[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:46:35 dist-test-slave-n326 krb5kdc[22459](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:46:35 dist-test-slave-n326 krb5kdc[22459](info): set up 2 sockets
Aug 11 02:46:35 dist-test-slave-n326 krb5kdc[22459](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 02:46:37.017385 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:46:37 dist-test-slave-n326 krb5kdc[22459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:46:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:46:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:46:37.152735 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:42095
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:36649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:42095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:37.275696 22477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:37.275969 22477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:37.276011 22477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:37.280246 22477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:46:37.280310 22477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:37.280330 22477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:37.280347 22477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:46:37.280362 22477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:46:37.285708 22477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:36649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:42095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:42095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:37.286780 22477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:37.287846 22477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:37.297633 22484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:37.297689 22483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:37.297633 22488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:37.297956 22477 server_base.cc:1034] running on GCE node
I20260811 02:46:37.298596 22477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:37.299865 22477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:37.301093 22477 hybrid_clock.cc:648] HybridClock initialized: now 1786416397301056 us; error 47 us; skew 500 ppm
Aug 11 02:46:37 dist-test-slave-n326 krb5kdc[22459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416397, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:37.304253 22477 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:46:37.305308 22477 webserver.cc:460] Webserver started at http://127.21.236.190:33647/ using document root <none> and password file <none>
I20260811 02:46:37.305912 22477 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:37.306008 22477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:37.306247 22477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:37.308470 22477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "83725248b5934d0b99204a494d928baa"
format_stamp: "Formatted at 2026-08-11 02:46:37 on dist-test-slave-n326"
server_key: "65061053f479fdb9befec0178fab0cca"
server_key_iv: "c747a7e4d450d2882efc4aae8e0671a8"
server_key_version: "encryptionkey@0"
I20260811 02:46:37.309072 22477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "83725248b5934d0b99204a494d928baa"
format_stamp: "Formatted at 2026-08-11 02:46:37 on dist-test-slave-n326"
server_key: "65061053f479fdb9befec0178fab0cca"
server_key_iv: "c747a7e4d450d2882efc4aae8e0671a8"
server_key_version: "encryptionkey@0"
I20260811 02:46:37.313114 22477 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:46:37.315850 22503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:37.317168 22477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:46:37.317337 22477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "83725248b5934d0b99204a494d928baa"
format_stamp: "Formatted at 2026-08-11 02:46:37 on dist-test-slave-n326"
server_key: "65061053f479fdb9befec0178fab0cca"
server_key_iv: "c747a7e4d450d2882efc4aae8e0671a8"
server_key_version: "encryptionkey@0"
I20260811 02:46:37.317487 22477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:37.335820 22477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:37.339037 22477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:37.339323 22477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:37.347659 22477 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:42095
I20260811 02:46:37.347718 22580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:42095 every 8 connection(s)
I20260811 02:46:37.348716 22477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:46:37.351162 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 22477
I20260811 02:46:37.351349 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:46:37.352069 22581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:37.352732 22450 external_mini_cluster.cc:1428] Setting key 4f2c3a79de53d79394d4ea3da58126e0
I20260811 02:46:37.358582 22581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa: Bootstrap starting.
I20260811 02:46:37.361425 22581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:37.362509 22581 log.cc:824] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:37.364919 22581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa: No bootstrap required, opened a new log
I20260811 02:46:37.369499 22581 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83725248b5934d0b99204a494d928baa" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42095 } }
I20260811 02:46:37.369737 22581 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:37.369817 22581 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83725248b5934d0b99204a494d928baa, State: Initialized, Role: FOLLOWER
I20260811 02:46:37.370306 22581 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83725248b5934d0b99204a494d928baa" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42095 } }
I20260811 02:46:37.370441 22581 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:46:37.370538 22581 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:46:37.370669 22581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [term 0 FOLLOWER]: Advancing to term 1
Aug 11 02:46:37 dist-test-slave-n326 krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:37.371811 22581 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83725248b5934d0b99204a494d928baa" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42095 } }
I20260811 02:46:37.372234 22581 leader_election.cc:302] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83725248b5934d0b99204a494d928baa; no voters:
I20260811 02:46:37.372654 22581 leader_election.cc:288] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:46:37.372802 22593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:37.373098 22593 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [term 1 LEADER]: Becoming Leader. State: Replica: 83725248b5934d0b99204a494d928baa, State: Running, Role: LEADER
I20260811 02:46:37.373469 22593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83725248b5934d0b99204a494d928baa" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42095 } }
I20260811 02:46:37.374315 22581 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:46:37.375416 22594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83725248b5934d0b99204a494d928baa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83725248b5934d0b99204a494d928baa" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42095 } } }
I20260811 02:46:37.375572 22594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [sys.catalog]: This master's current role is: LEADER
I20260811 02:46:37.375900 22596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83725248b5934d0b99204a494d928baa. Latest consensus state: current_term: 1 leader_uuid: "83725248b5934d0b99204a494d928baa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83725248b5934d0b99204a494d928baa" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42095 } } }
I20260811 02:46:37.376062 22596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa [sys.catalog]: This master's current role is: LEADER
I20260811 02:46:37.375998 22607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:46:37.379994 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:37.355978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38528 (local address 127.21.236.190:42095)
0811 02:46:37.356514 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:37.356536 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:37.357600 (+ 1064us) server_negotiation.cc:407] Connection header received
0811 02:46:37.360985 (+ 3385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:37.361009 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:37.361488 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:37.361891 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:37.364094 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:37.365731 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:37.366925 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:37.367761 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:37.367983 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:37.372148 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:37.372155 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:37.372173 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:37.372208 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:37.374743 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:37.375612 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:37.375620 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:37.375626 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:37.375801 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:37.376295 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:37.376298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:37.376301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:37.376908 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:37.377113 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:37.378765 (+ 1652us) server_negotiation.cc:299] Negotiation successful
0811 02:46:37.379048 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":138,"threads_started":1}
I20260811 02:46:37.380357 22607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:46:37.386633 22607 catalog_manager.cc:1347] Generated new cluster ID: 64afa9b22f2e411c90dc00042dc0ccbc
I20260811 02:46:37.386741 22607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:46:37.400095 22607 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:46:37.401404 22607 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:46:37.413511 22607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83725248b5934d0b99204a494d928baa: Generated new TSK 0
I20260811 02:46:37.414299 22607 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:46:37.459522 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:42095
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:36649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:37.624799 22627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:37.625087 22627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:37.625133 22627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:37.629439 22627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:37.629515 22627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:37.629591 22627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:46:37.635347 22627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:36649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:42095
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:37.636469 22627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:37.637679 22627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:37.650650 22635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:37.650676 22638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:37.650666 22636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:37.651038 22627 server_base.cc:1034] running on GCE node
I20260811 02:46:37.651525 22627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:37.652118 22627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:37.653313 22627 hybrid_clock.cc:648] HybridClock initialized: now 1786416397653296 us; error 38 us; skew 500 ppm
Aug 11 02:46:37 dist-test-slave-n326 krb5kdc[22459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416397, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:37.656903 22627 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:46:37.657714 22627 webserver.cc:460] Webserver started at http://127.21.236.129:41395/ using document root <none> and password file <none>
I20260811 02:46:37.658241 22627 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:37.658303 22627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:37.658537 22627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:37.660702 22627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "dc27e5cbadca4b6abb666aae260165bc"
format_stamp: "Formatted at 2026-08-11 02:46:37 on dist-test-slave-n326"
server_key: "5ceffa63bace46cbb34151a3fb8bd4ec"
server_key_iv: "70e453f5ebca8a4cd9c96e31d6e1ae49"
server_key_version: "encryptionkey@0"
I20260811 02:46:37.661510 22627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "dc27e5cbadca4b6abb666aae260165bc"
format_stamp: "Formatted at 2026-08-11 02:46:37 on dist-test-slave-n326"
server_key: "5ceffa63bace46cbb34151a3fb8bd4ec"
server_key_iv: "70e453f5ebca8a4cd9c96e31d6e1ae49"
server_key_version: "encryptionkey@0"
I20260811 02:46:37.665966 22627 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:46:37.668785 22650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:37.670202 22627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:46:37.670310 22627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "dc27e5cbadca4b6abb666aae260165bc"
format_stamp: "Formatted at 2026-08-11 02:46:37 on dist-test-slave-n326"
server_key: "5ceffa63bace46cbb34151a3fb8bd4ec"
server_key_iv: "70e453f5ebca8a4cd9c96e31d6e1ae49"
server_key_version: "encryptionkey@0"
I20260811 02:46:37.670468 22627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:37.682327 22627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:37.685873 22627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:37.686154 22627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:37.686966 22627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:46:37.688411 22627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:37.688474 22627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:37.688525 22627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:37.688548 22627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:37.707965 22627 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:44673
I20260811 02:46:37.708047 22823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:44673 every 8 connection(s)
I20260811 02:46:37.709059 22627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:46:37.719503 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 22627
I20260811 02:46:37.719642 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:46:37.719935 22450 external_mini_cluster.cc:1428] Setting key 76c5d04990e46ce1996b7b89d1a1fec6
Aug 11 02:46:37 dist-test-slave-n326 krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416397, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:37.726934 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:37.711370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:44005 (local address 127.21.236.190:42095)
0811 02:46:37.711516 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:37.711521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:37.712788 (+ 1267us) server_negotiation.cc:407] Connection header received
0811 02:46:37.714473 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:37.714477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:37.714538 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:37.714645 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:37.716264 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:37.716819 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:37.718213 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:37.718471 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:37.718588 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:37.722334 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:37.722338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:37.722340 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:37.722363 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:37.723977 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:37.725025 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:37.725032 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:37.725035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:37.725141 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:37.725535 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:37.725540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:37.725542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:37.725807 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:37.725939 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:37.726513 (+ 574us) server_negotiation.cc:299] Negotiation successful
0811 02:46:37.726678 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:46:37.727880 22827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:37.711871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:42095 (local address 127.21.236.129:44005)
0811 02:46:37.712616 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:37.712663 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:37.714234 (+ 1571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:37.714832 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:37.714839 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:37.715483 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:37.716099 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:37.716115 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:37.716962 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:37.716965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:37.718080 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:37.718090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:37.718636 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:37.718640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:37.718874 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:37.719775 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:37.719793 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:37.722089 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:37.724119 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:37.724130 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:37.724151 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:37.724778 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:37.725260 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:37.725263 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:37.725265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:37.725426 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:37.725947 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:37.725952 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:37.726230 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 02:46:37.726418 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 02:46:37.726714 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":184,"threads_started":1}
I20260811 02:46:37.729417 22824 heartbeater.cc:347] Connected to a master server at 127.21.236.190:42095
I20260811 02:46:37.729717 22824 heartbeater.cc:464] Registering TS with master...
I20260811 02:46:37.730312 22824 heartbeater.cc:511] Master 127.21.236.190:42095 requested a full tablet report, sending...
I20260811 02:46:37.732038 22529 ts_manager.cc:194] Registered new tserver with Master: dc27e5cbadca4b6abb666aae260165bc (127.21.236.129:44673)
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
I20260811 02:46:37.733356 22529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:44005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:46:37.758767 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:42095
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:36649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:37.919313 22834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:37.919579 22834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:37.919658 22834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:37.924003 22834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:37.924072 22834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:37.924176 22834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:46:37.929445 22834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:36649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:42095
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:37.930502 22834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:37.931594 22834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:37.941398 22844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:37.941411 22842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:37.941614 22834 server_base.cc:1034] running on GCE node
W20260811 02:46:37.941509 22847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:37.942148 22834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:37.942762 22834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:37.943941 22834 hybrid_clock.cc:648] HybridClock initialized: now 1786416397943895 us; error 59 us; skew 500 ppm
Aug 11 02:46:37 dist-test-slave-n326 krb5kdc[22459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416397, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:37.947234 22834 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:46:37.948289 22834 webserver.cc:460] Webserver started at http://127.21.236.130:43995/ using document root <none> and password file <none>
I20260811 02:46:37.948915 22834 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:37.949020 22834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:37.949271 22834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:37.951495 22834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "f5a64f89795d4975a144c0fe65506576"
format_stamp: "Formatted at 2026-08-11 02:46:37 on dist-test-slave-n326"
server_key: "3a62d725b85b210440cfb27f00cac34e"
server_key_iv: "ac51924ffb383e634e05de22689d0f47"
server_key_version: "encryptionkey@0"
I20260811 02:46:37.952092 22834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "f5a64f89795d4975a144c0fe65506576"
format_stamp: "Formatted at 2026-08-11 02:46:37 on dist-test-slave-n326"
server_key: "3a62d725b85b210440cfb27f00cac34e"
server_key_iv: "ac51924ffb383e634e05de22689d0f47"
server_key_version: "encryptionkey@0"
I20260811 02:46:37.956233 22834 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:46:37.958845 22857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:37.960374 22834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:46:37.960543 22834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "f5a64f89795d4975a144c0fe65506576"
format_stamp: "Formatted at 2026-08-11 02:46:37 on dist-test-slave-n326"
server_key: "3a62d725b85b210440cfb27f00cac34e"
server_key_iv: "ac51924ffb383e634e05de22689d0f47"
server_key_version: "encryptionkey@0"
I20260811 02:46:37.960680 22834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:37.984321 22834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:37.988163 22834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:37.988391 22834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:37.989049 22834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:46:37.989946 22834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:37.990020 22834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:37.990110 22834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:37.990162 22834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:38.000702 22834 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:46573
I20260811 02:46:38.000722 23040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:46573 every 8 connection(s)
I20260811 02:46:38.001662 22834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:46:38.007931 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 22834
I20260811 02:46:38.008064 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:46:38.008378 22450 external_mini_cluster.cc:1428] Setting key 1048fd0f92710b2e6ae598552ae0e964
Aug 11 02:46:38 dist-test-slave-n326 krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416397, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:38.016832 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:38.003688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:40747 (local address 127.21.236.190:42095)
0811 02:46:38.003835 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:38.003839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:38.004745 (+ 906us) server_negotiation.cc:407] Connection header received
0811 02:46:38.005975 (+ 1230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:38.005978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:38.006028 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:38.006113 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:38.007381 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:38.007910 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:38.009196 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:38.009428 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:38.009544 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:38.012541 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:38.012545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:38.012549 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:38.012584 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:38.014442 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:38.015119 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:38.015123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:38.015125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:38.015219 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:38.015651 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:38.015653 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:38.015655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:38.015831 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:38.015908 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:38.016602 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 02:46:38.016709 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 02:46:38.018198 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:38.004091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:42095 (local address 127.21.236.130:40747)
0811 02:46:38.004587 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:38.004632 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:38.005777 (+ 1145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:38.006278 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:38.006284 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:38.006673 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:38.007247 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:38.007263 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:38.008072 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:38.008079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:38.009061 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:38.009071 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:38.009604 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:38.009609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:38.009776 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:38.010401 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:38.010418 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:38.012371 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:38.014558 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:38.014566 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:38.014583 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:38.014968 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:38.015321 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:38.015327 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:38.015332 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:38.015550 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:38.015948 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:38.015957 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:38.016318 (+ 361us) client_negotiation.cc:769] Sending connection context
0811 02:46:38.016552 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 02:46:38.016956 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":144,"threads_started":1}
I20260811 02:46:38.019543 23041 heartbeater.cc:347] Connected to a master server at 127.21.236.190:42095
I20260811 02:46:38.019840 23041 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
I20260811 02:46:38.020457 23041 heartbeater.cc:511] Master 127.21.236.190:42095 requested a full tablet report, sending...
I20260811 02:46:38.021787 22529 ts_manager.cc:194] Registered new tserver with Master: f5a64f89795d4975a144c0fe65506576 (127.21.236.130:46573)
I20260811 02:46:38.022353 22529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:40747
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:46:38.040252 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:42095
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:36649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:38.157780 23048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:38.158031 23048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:38.158074 23048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:38.162112 23048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:38.162170 23048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:38.162257 23048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:46:38.167732 23048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:36649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:42095
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:38.168766 23048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:38.169783 23048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:38.180073 23062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:38.180117 23060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:38.180357 23048 server_base.cc:1034] running on GCE node
W20260811 02:46:38.180073 23058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:38.180816 23048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:38.181524 23048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:38.182691 23048 hybrid_clock.cc:648] HybridClock initialized: now 1786416398182675 us; error 28 us; skew 500 ppm
Aug 11 02:46:38 dist-test-slave-n326 krb5kdc[22459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416398, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:38.185489 23048 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:46:38.186638 23048 webserver.cc:460] Webserver started at http://127.21.236.131:43373/ using document root <none> and password file <none>
I20260811 02:46:38.187577 23048 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:38.187675 23048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:38.187938 23048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:38.191195 23048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "936ebdb1811d4b21befa7f3090555274"
format_stamp: "Formatted at 2026-08-11 02:46:38 on dist-test-slave-n326"
server_key: "71cd8397bcc616899323a586c4c55b04"
server_key_iv: "bbee3d60732136174552a0f6ff53caa9"
server_key_version: "encryptionkey@0"
I20260811 02:46:38.191960 23048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "936ebdb1811d4b21befa7f3090555274"
format_stamp: "Formatted at 2026-08-11 02:46:38 on dist-test-slave-n326"
server_key: "71cd8397bcc616899323a586c4c55b04"
server_key_iv: "bbee3d60732136174552a0f6ff53caa9"
server_key_version: "encryptionkey@0"
I20260811 02:46:38.197633 23048 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 02:46:38.200624 23073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:38.202209 23048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:46:38.202332 23048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "936ebdb1811d4b21befa7f3090555274"
format_stamp: "Formatted at 2026-08-11 02:46:38 on dist-test-slave-n326"
server_key: "71cd8397bcc616899323a586c4c55b04"
server_key_iv: "bbee3d60732136174552a0f6ff53caa9"
server_key_version: "encryptionkey@0"
I20260811 02:46:38.202450 23048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:38.221596 23048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:38.224767 23048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:38.224932 23048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:38.225504 23048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:46:38.226323 23048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:38.226373 23048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:38.226414 23048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:38.226430 23048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:38.236740 23048 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:45229
I20260811 02:46:38.236779 23259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:45229 every 8 connection(s)
I20260811 02:46:38.237696 23048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:46:38.238920 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 23048
I20260811 02:46:38.239089 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:46:38.239423 22450 external_mini_cluster.cc:1428] Setting key 5be7a9bd96ec3ca3b9098faceeef712e
Aug 11 02:46:38 dist-test-slave-n326 krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416398, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:38.252789 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:38.239996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:38223 (local address 127.21.236.190:42095)
0811 02:46:38.240176 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:38.240181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:38.241185 (+ 1004us) server_negotiation.cc:407] Connection header received
0811 02:46:38.242923 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:38.242926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:38.242995 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:38.243104 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:38.244365 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:38.244963 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:38.246201 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:38.246463 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:38.246596 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:38.249224 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:38.249228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:38.249230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:38.249255 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:38.250728 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:38.251322 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:38.251325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:38.251327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:38.251406 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:38.251724 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:38.251726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:38.251727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:38.251897 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:38.251972 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:38.252476 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 02:46:38.252599 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:46:38.253880 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:38.240428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:42095 (local address 127.21.236.131:38223)
0811 02:46:38.241020 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:38.241077 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:38.242706 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:38.243300 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:38.243307 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:38.243709 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:38.244213 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:38.244227 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:38.245102 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:38.245106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:38.246062 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:38.246072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:38.246629 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:38.246633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:38.246824 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:38.247426 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:38.247444 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:38.249062 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:38.250853 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:38.250857 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:38.250868 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:38.251220 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:38.251492 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:38.251494 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:38.251496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:38.251633 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:38.252014 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:38.252019 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:38.252250 (+ 231us) client_negotiation.cc:769] Sending connection context
0811 02:46:38.252422 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 02:46:38.252805 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":153,"threads_started":1}
I20260811 02:46:38.255095 23260 heartbeater.cc:347] Connected to a master server at 127.21.236.190:42095
I20260811 02:46:38.255340 23260 heartbeater.cc:464] Registering TS with master...
I20260811 02:46:38.255937 23260 heartbeater.cc:511] Master 127.21.236.190:42095 requested a full tablet report, sending...
I20260811 02:46:38.257040 22529 ts_manager.cc:194] Registered new tserver with Master: 936ebdb1811d4b21befa7f3090555274 (127.21.236.131:45229)
I20260811 02:46:38.257611 22529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:38223
I20260811 02:46:38.266436 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 02:46:38 dist-test-slave-n326 krb5kdc[22459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416398, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:46:38 dist-test-slave-n326 krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416398, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.236.129@KRBTEST.COM
I20260811 02:46:38.297461 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:38.285905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47538 (local address 127.21.236.129:44673)
0811 02:46:38.286320 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:38.286327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:38.286353 (+ 26us) server_negotiation.cc:407] Connection header received
0811 02:46:38.286441 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:38.286447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:38.286646 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:38.286781 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:38.287618 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:38.288545 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:38.289499 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:38.289944 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:38.290044 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:38.292891 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:38.292904 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:38.292918 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:38.292947 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:38.295285 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:38.295796 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:38.295802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:38.295804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:38.295901 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:38.296258 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:38.296261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:38.296262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:38.296595 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:38.296782 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:38.297073 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 02:46:38.297273 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":174,"threads_started":1}
W20260811 02:46:38.298528 22687 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47538
Aug 11 02:46:38 dist-test-slave-n326 krb5kdc[22459](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:46:38 dist-test-slave-n326 krb5kdc[22459](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.129@KRBTEST.COM
I20260811 02:46:38.330049 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:38.320042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47550 (local address 127.21.236.129:44673)
0811 02:46:38.320241 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:38.320245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:38.320264 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:46:38.320447 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:38.320451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:38.320512 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:38.320593 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:38.321553 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:38.322157 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:38.323350 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:38.323621 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:38.323721 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:38.326559 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:38.326563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:38.326565 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:38.326592 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:38.328200 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:38.328678 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:38.328682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:38.328683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:38.328842 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:38.329133 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:38.329135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:38.329136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:38.329312 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:38.329424 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:38.329698 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 02:46:38.329858 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 02:46:38.331436 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 22627
I20260811 02:46:38.346513 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 22834
I20260811 02:46:38.376852 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 23048
I20260811 02:46:38.390336 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 22477
2026-08-11T02:46:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2705 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:46:38 dist-test-slave-n326 krb5kdc[23285](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:46:38 dist-test-slave-n326 krb5kdc[23285](info): set up 2 sockets
Aug 11 02:46:38 dist-test-slave-n326 krb5kdc[23285](info): commencing operation
krb5kdc: starting...
W20260811 02:46:39.686117 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:46:39 dist-test-slave-n326 krb5kdc[23285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:46:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:46:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:46:39.815029 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:34047
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:34047
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:39.928334 23310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:39.928643 23310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:39.928723 23310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:39.932732 23310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:46:39.932817 23310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:39.932855 23310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:39.932915 23310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:46:39.932952 23310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:46:39.938171 23310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:34047
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:34047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:39.939334 23310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:39.940398 23310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:39.949604 23318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:39.949594 23316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:39.949571 23315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:39.950376 23310 server_base.cc:1034] running on GCE node
I20260811 02:46:39.950980 23310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:39.952167 23310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:39.953365 23310 hybrid_clock.cc:648] HybridClock initialized: now 1786416399953338 us; error 48 us; skew 500 ppm
Aug 11 02:46:39 dist-test-slave-n326 krb5kdc[23285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416399, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:39.959599 23310 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:46:39.960603 23310 webserver.cc:460] Webserver started at http://127.21.236.190:43787/ using document root <none> and password file <none>
I20260811 02:46:39.961213 23310 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:39.961311 23310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:39.961575 23310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:39.963747 23310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "9cb4ced772e24633b9dafe369674b7b0"
format_stamp: "Formatted at 2026-08-11 02:46:39 on dist-test-slave-n326"
server_key: "9105b6f727d286e016e6619e25d3de5c"
server_key_iv: "8651c8f31763fb4627156a65caa10c8c"
server_key_version: "encryptionkey@0"
I20260811 02:46:39.964381 23310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "9cb4ced772e24633b9dafe369674b7b0"
format_stamp: "Formatted at 2026-08-11 02:46:39 on dist-test-slave-n326"
server_key: "9105b6f727d286e016e6619e25d3de5c"
server_key_iv: "8651c8f31763fb4627156a65caa10c8c"
server_key_version: "encryptionkey@0"
I20260811 02:46:39.968588 23310 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:46:39.971153 23328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:39.972424 23310 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:46:39.972591 23310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "9cb4ced772e24633b9dafe369674b7b0"
format_stamp: "Formatted at 2026-08-11 02:46:39 on dist-test-slave-n326"
server_key: "9105b6f727d286e016e6619e25d3de5c"
server_key_iv: "8651c8f31763fb4627156a65caa10c8c"
server_key_version: "encryptionkey@0"
I20260811 02:46:39.972712 23310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:39.988723 23310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:39.992218 23310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:39.992437 23310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:39.999653 23310 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:34047
I20260811 02:46:39.999673 23411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:34047 every 8 connection(s)
I20260811 02:46:40.000670 23310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:46:40.001924 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 23310
I20260811 02:46:40.002025 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:46:40.002512 22450 external_mini_cluster.cc:1428] Setting key bb2f9cdd0df8acca3ccc4bb40ff9f476
I20260811 02:46:40.004092 23412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:40.011135 23412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0: Bootstrap starting.
Aug 11 02:46:40 dist-test-slave-n326 krb5kdc[23285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:40.013908 23412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:40.014819 23412 log.cc:824] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:40.016821 23412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0: No bootstrap required, opened a new log
I20260811 02:46:40.017942 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:40.004182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38908 (local address 127.21.236.190:34047)
0811 02:46:40.004643 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:40.004663 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:40.004700 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:46:40.005375 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:40.005391 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:40.005716 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:40.006009 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:40.007011 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.008114 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.009297 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.009987 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.010175 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:40.013243 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:40.013249 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:40.013262 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:40.013289 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:40.015183 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:40.015638 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:40.015644 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:40.015649 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:40.015769 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:40.016072 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:40.016075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:40.016077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:40.016469 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:40.016674 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:40.016949 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 02:46:40.017248 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":153,"threads_started":1}
I20260811 02:46:40.020457 23412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb4ced772e24633b9dafe369674b7b0" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 34047 } }
I20260811 02:46:40.020803 23412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:40.020884 23412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb4ced772e24633b9dafe369674b7b0, State: Initialized, Role: FOLLOWER
I20260811 02:46:40.021385 23412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb4ced772e24633b9dafe369674b7b0" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 34047 } }
I20260811 02:46:40.021574 23412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:46:40.021648 23412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:46:40.021751 23412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:40.022779 23412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb4ced772e24633b9dafe369674b7b0" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 34047 } }
I20260811 02:46:40.023195 23412 leader_election.cc:302] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cb4ced772e24633b9dafe369674b7b0; no voters:
I20260811 02:46:40.023545 23412 leader_election.cc:288] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:46:40.023965 23420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:40.024158 23420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb4ced772e24633b9dafe369674b7b0, State: Running, Role: LEADER
I20260811 02:46:40.024451 23420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb4ced772e24633b9dafe369674b7b0" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 34047 } }
I20260811 02:46:40.024896 23412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:46:40.025741 23420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cb4ced772e24633b9dafe369674b7b0. Latest consensus state: current_term: 1 leader_uuid: "9cb4ced772e24633b9dafe369674b7b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb4ced772e24633b9dafe369674b7b0" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 34047 } } }
I20260811 02:46:40.025884 23420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [sys.catalog]: This master's current role is: LEADER
I20260811 02:46:40.026295 23432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:46:40.026276 23422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cb4ced772e24633b9dafe369674b7b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb4ced772e24633b9dafe369674b7b0" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 34047 } } }
I20260811 02:46:40.026399 23422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0 [sys.catalog]: This master's current role is: LEADER
I20260811 02:46:40.030049 23432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:46:40.040736 23432 catalog_manager.cc:1347] Generated new cluster ID: a78f6adc3bfa4242ad4c6e27741d737f
I20260811 02:46:40.040856 23432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:46:40.058807 23432 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:46:40.060233 23432 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:46:40.066087 23432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cb4ced772e24633b9dafe369674b7b0: Generated new TSK 0
I20260811 02:46:40.066924 23432 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:46:40.122249 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:34047
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:40.260111 23452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:40.260401 23452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:40.260448 23452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:40.264550 23452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:40.264621 23452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:40.264681 23452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:46:40.270028 23452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:34047
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:40.271099 23452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:40.272111 23452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:40.281999 23465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:40.282035 23461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:40.282001 23463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:40.282457 23452 server_base.cc:1034] running on GCE node
I20260811 02:46:40.282840 23452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:40.283532 23452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:40.284704 23452 hybrid_clock.cc:648] HybridClock initialized: now 1786416400284689 us; error 25 us; skew 500 ppm
Aug 11 02:46:40 dist-test-slave-n326 krb5kdc[23285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416400, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:40.287530 23452 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:46:40.288401 23452 webserver.cc:460] Webserver started at http://127.21.236.129:44923/ using document root <none> and password file <none>
I20260811 02:46:40.288986 23452 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:40.289049 23452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:40.289302 23452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:40.291558 23452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "6bfb3efb6b094a989c8e245a35b467ff"
format_stamp: "Formatted at 2026-08-11 02:46:40 on dist-test-slave-n326"
server_key: "115aeec338906ca8b88078e6808008d7"
server_key_iv: "290957b73f60f20922a4c17ee68aa07e"
server_key_version: "encryptionkey@0"
I20260811 02:46:40.292135 23452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "6bfb3efb6b094a989c8e245a35b467ff"
format_stamp: "Formatted at 2026-08-11 02:46:40 on dist-test-slave-n326"
server_key: "115aeec338906ca8b88078e6808008d7"
server_key_iv: "290957b73f60f20922a4c17ee68aa07e"
server_key_version: "encryptionkey@0"
I20260811 02:46:40.296054 23452 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:46:40.298602 23474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:40.299949 23452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:46:40.300099 23452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "6bfb3efb6b094a989c8e245a35b467ff"
format_stamp: "Formatted at 2026-08-11 02:46:40 on dist-test-slave-n326"
server_key: "115aeec338906ca8b88078e6808008d7"
server_key_iv: "290957b73f60f20922a4c17ee68aa07e"
server_key_version: "encryptionkey@0"
I20260811 02:46:40.300216 23452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:40.335996 23452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:40.339428 23452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:40.339651 23452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:40.340293 23452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:46:40.341185 23452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:40.341256 23452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:40.341328 23452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:40.341391 23452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:40.351601 23452 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:34085
I20260811 02:46:40.351635 23652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:34085 every 8 connection(s)
I20260811 02:46:40.352566 23452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:46:40.362039 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 23452
I20260811 02:46:40.362185 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
Aug 11 02:46:40 dist-test-slave-n326 krb5kdc[23285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416400, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:40.362509 22450 external_mini_cluster.cc:1428] Setting key 3b70c4e912ba468292aa52ccaaaa22fd
I20260811 02:46:40.367967 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:40.354611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:56029 (local address 127.21.236.190:34047)
0811 02:46:40.354770 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:40.354773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:40.355643 (+ 870us) server_negotiation.cc:407] Connection header received
0811 02:46:40.356515 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:40.356518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:40.356569 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:40.356642 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:40.357877 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.358420 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.359637 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.359873 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.359967 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:40.363237 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:40.363242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:40.363246 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:40.363277 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:40.365394 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:40.365982 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:40.365989 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:40.365995 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:40.366107 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:40.366500 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:40.366506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:40.366511 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:40.366827 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:40.366955 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:40.367635 (+ 680us) server_negotiation.cc:299] Negotiation successful
0811 02:46:40.367788 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:46:40.368909 23657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:40.354940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:34047 (local address 127.21.236.129:56029)
0811 02:46:40.355484 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:40.355529 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:40.356318 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:40.356855 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:40.356862 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:40.357236 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:40.357740 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:40.357754 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.358554 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.358557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:40.359519 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:40.359530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.360063 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.360067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:40.360246 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:40.361055 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:40.361074 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:40.362950 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:40.365544 (+ 2594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:40.365550 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:40.365565 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:40.365864 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:40.366216 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:40.366218 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:40.366220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:40.366388 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:40.366959 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:40.366970 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:40.367351 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 02:46:40.367578 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 02:46:40.367966 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":175,"threads_started":1}
I20260811 02:46:40.370436 23653 heartbeater.cc:347] Connected to a master server at 127.21.236.190:34047
I20260811 02:46:40.370779 23653 heartbeater.cc:464] Registering TS with master...
I20260811 02:46:40.371608 23653 heartbeater.cc:511] Master 127.21.236.190:34047 requested a full tablet report, sending...
I20260811 02:46:40.373766 23359 ts_manager.cc:194] Registered new tserver with Master: 6bfb3efb6b094a989c8e245a35b467ff (127.21.236.129:34085)
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
I20260811 02:46:40.375603 23359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:56029
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:46:40.396495 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:34047
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:40.514837 23661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:40.515193 23661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:40.515247 23661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:40.519410 23661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:40.519474 23661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:40.519537 23661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:46:40.524854 23661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:34047
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:40.526054 23661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:40.527320 23661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:40.537072 23668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:40.537134 23669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:40.537178 23671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:40.537681 23661 server_base.cc:1034] running on GCE node
I20260811 02:46:40.538151 23661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:40.538867 23661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:40.540064 23661 hybrid_clock.cc:648] HybridClock initialized: now 1786416400540013 us; error 58 us; skew 500 ppm
Aug 11 02:46:40 dist-test-slave-n326 krb5kdc[23285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416400, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:40.543417 23661 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:46:40.544320 23661 webserver.cc:460] Webserver started at http://127.21.236.130:33205/ using document root <none> and password file <none>
I20260811 02:46:40.544912 23661 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:40.545011 23661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:40.545251 23661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:40.547396 23661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "1a6dc1debab84c0d91dc83c1b41c8575"
format_stamp: "Formatted at 2026-08-11 02:46:40 on dist-test-slave-n326"
server_key: "ebcd079ac726a872584c3f67e9d87122"
server_key_iv: "543879f9b3ddfeaa2062ea3aaef2e8ea"
server_key_version: "encryptionkey@0"
I20260811 02:46:40.547963 23661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "1a6dc1debab84c0d91dc83c1b41c8575"
format_stamp: "Formatted at 2026-08-11 02:46:40 on dist-test-slave-n326"
server_key: "ebcd079ac726a872584c3f67e9d87122"
server_key_iv: "543879f9b3ddfeaa2062ea3aaef2e8ea"
server_key_version: "encryptionkey@0"
I20260811 02:46:40.551721 23661 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:46:40.554186 23684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:40.555416 23661 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 02:46:40.555552 23661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "1a6dc1debab84c0d91dc83c1b41c8575"
format_stamp: "Formatted at 2026-08-11 02:46:40 on dist-test-slave-n326"
server_key: "ebcd079ac726a872584c3f67e9d87122"
server_key_iv: "543879f9b3ddfeaa2062ea3aaef2e8ea"
server_key_version: "encryptionkey@0"
I20260811 02:46:40.555666 23661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:40.580878 23661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:40.583900 23661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:40.584121 23661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:40.584720 23661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:46:40.585594 23661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:40.585667 23661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:40.585740 23661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:40.585778 23661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:40.594551 23661 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:33673
I20260811 02:46:40.594638 23853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:33673 every 8 connection(s)
I20260811 02:46:40.595557 23661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
Aug 11 02:46:40 dist-test-slave-n326 krb5kdc[23285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416400, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:40.604799 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 23661
I20260811 02:46:40.604943 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:46:40.605247 22450 external_mini_cluster.cc:1428] Setting key c1e72db0ed0c82587266154dc3f25b08
I20260811 02:46:40.609028 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:40.597428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:60313 (local address 127.21.236.190:34047)
0811 02:46:40.597610 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:40.597615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:40.598451 (+ 836us) server_negotiation.cc:407] Connection header received
0811 02:46:40.599358 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:40.599360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:40.599408 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:40.599481 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:40.600634 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.601164 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.602255 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.602481 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.602572 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:40.605263 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:40.605269 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:40.605272 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:40.605304 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:40.606879 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:40.607468 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:40.607471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:40.607473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:40.607561 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:40.607926 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:40.607930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:40.607931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:40.608123 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:40.608231 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:40.608734 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 02:46:40.608860 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 02:46:40.609834 23859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:40.597831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:34047 (local address 127.21.236.130:60313)
0811 02:46:40.598286 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:40.598329 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:40.599163 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:40.599655 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:40.599661 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:40.600007 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:40.600471 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:40.600483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.601272 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.601274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:40.602139 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:40.602148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.602620 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.602623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:40.602757 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:40.603455 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:40.603472 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:40.605075 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:40.607002 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:40.607009 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:40.607023 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:40.607362 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:40.607672 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:40.607675 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:40.607677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:40.607823 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:40.608233 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:40.608241 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:40.608486 (+ 245us) client_negotiation.cc:769] Sending connection context
0811 02:46:40.608672 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 02:46:40.608996 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":113,"threads_started":1}
I20260811 02:46:40.611105 23856 heartbeater.cc:347] Connected to a master server at 127.21.236.190:34047
I20260811 02:46:40.611356 23856 heartbeater.cc:464] Registering TS with master...
I20260811 02:46:40.611865 23856 heartbeater.cc:511] Master 127.21.236.190:34047 requested a full tablet report, sending...
I20260811 02:46:40.612933 23359 ts_manager.cc:194] Registered new tserver with Master: 1a6dc1debab84c0d91dc83c1b41c8575 (127.21.236.130:33673)
I20260811 02:46:40.613510 23359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:60313
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:46:40.635156 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:34047
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:40.756910 23867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:40.757213 23867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:40.757261 23867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:40.761505 23867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:40.761605 23867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:40.761672 23867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:46:40.767053 23867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:34047
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:40.768234 23867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:40.769269 23867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:40.780359 23887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:40.780366 23880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:40.780366 23885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:40.780728 23867 server_base.cc:1034] running on GCE node
I20260811 02:46:40.781252 23867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:40.782042 23867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:40.783176 23867 hybrid_clock.cc:648] HybridClock initialized: now 1786416400783159 us; error 30 us; skew 500 ppm
Aug 11 02:46:40 dist-test-slave-n326 krb5kdc[23285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416400, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:40.786289 23867 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:46:40.787326 23867 webserver.cc:460] Webserver started at http://127.21.236.131:39101/ using document root <none> and password file <none>
I20260811 02:46:40.787925 23867 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:40.787988 23867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:40.788166 23867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:40.790354 23867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "032189d726df457db4a44ef9c42a4d64"
format_stamp: "Formatted at 2026-08-11 02:46:40 on dist-test-slave-n326"
server_key: "6799f710d128c3faeac3e4b27efc7199"
server_key_iv: "f366ff8944798a5c7de3de0d6a80e6c1"
server_key_version: "encryptionkey@0"
I20260811 02:46:40.791117 23867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "032189d726df457db4a44ef9c42a4d64"
format_stamp: "Formatted at 2026-08-11 02:46:40 on dist-test-slave-n326"
server_key: "6799f710d128c3faeac3e4b27efc7199"
server_key_iv: "f366ff8944798a5c7de3de0d6a80e6c1"
server_key_version: "encryptionkey@0"
I20260811 02:46:40.795053 23867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:46:40.797684 23894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:40.799181 23867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 02:46:40.799288 23867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "032189d726df457db4a44ef9c42a4d64"
format_stamp: "Formatted at 2026-08-11 02:46:40 on dist-test-slave-n326"
server_key: "6799f710d128c3faeac3e4b27efc7199"
server_key_iv: "f366ff8944798a5c7de3de0d6a80e6c1"
server_key_version: "encryptionkey@0"
I20260811 02:46:40.799381 23867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:40.812601 23867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:40.815964 23867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:40.816135 23867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:40.816715 23867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:46:40.817530 23867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:40.817581 23867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:40.817624 23867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:40.817641 23867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:40.826925 23867 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:37691
I20260811 02:46:40.826990 24089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:37691 every 8 connection(s)
I20260811 02:46:40.827930 23867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:46:40.832962 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 23867
I20260811 02:46:40.833148 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:46:40.833472 22450 external_mini_cluster.cc:1428] Setting key 4db3dd3afb02e9d0c0e9ce9854d65bb3
Aug 11 02:46:40 dist-test-slave-n326 krb5kdc[23285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416400, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:40.841917 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:40.830012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:58683 (local address 127.21.236.190:34047)
0811 02:46:40.830153 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:40.830164 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:40.831010 (+ 846us) server_negotiation.cc:407] Connection header received
0811 02:46:40.831976 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:40.831979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:40.832042 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:40.832157 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:40.833499 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.834090 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.835331 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.835557 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.835670 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:40.838247 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:40.838250 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:40.838252 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:40.838278 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:40.839811 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:40.840318 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:40.840321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:40.840322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:40.840404 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:40.840733 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:40.840735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:40.840736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:40.840920 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:40.841008 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:40.841673 (+ 665us) server_negotiation.cc:299] Negotiation successful
0811 02:46:40.841790 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:46:40.842905 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:40.830374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:34047 (local address 127.21.236.131:58683)
0811 02:46:40.830852 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:40.830896 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:40.831759 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:40.832341 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:40.832351 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:40.832756 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:40.833349 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:40.833366 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.834224 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.834228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:40.835202 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:40.835213 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.835725 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.835729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:40.835915 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:40.836502 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:40.836519 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:40.838063 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:40.839928 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:40.839933 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:40.839947 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:40.840221 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:40.840487 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:40.840489 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:40.840490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:40.840645 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:40.841120 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:40.841126 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:40.841420 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 02:46:40.841624 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 02:46:40.841927 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":127,"threads_started":1}
I20260811 02:46:40.844288 24090 heartbeater.cc:347] Connected to a master server at 127.21.236.190:34047
I20260811 02:46:40.844574 24090 heartbeater.cc:464] Registering TS with master...
I20260811 02:46:40.845151 24090 heartbeater.cc:511] Master 127.21.236.190:34047 requested a full tablet report, sending...
I20260811 02:46:40.846310 23359 ts_manager.cc:194] Registered new tserver with Master: 032189d726df457db4a44ef9c42a4d64 (127.21.236.131:37691)
I20260811 02:46:40.847345 23359 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:58683
I20260811 02:46:40.848054 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:46:40.859023 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:40.851694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38922 (local address 127.21.236.190:34047)
0811 02:46:40.851824 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:40.851828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:40.851959 (+ 131us) server_negotiation.cc:407] Connection header received
0811 02:46:40.852050 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:40.852053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:40.852096 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:40.852172 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:40.852878 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.853428 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.854311 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.854557 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.854641 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:40.856046 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:40.856049 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:40.856051 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:40.856074 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:40.857305 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:40.857739 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:40.857741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:40.857743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:40.857821 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:40.858110 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:40.858113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:40.858114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:40.858350 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:40.858480 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:40.858772 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 02:46:40.858877 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:46:40.864678 23351 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:46:40.867754 23351 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:46:40.883633 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:40.877751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45066 (local address 127.21.236.129:34085)
0811 02:46:40.878131 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:40.878138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:40.878163 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:46:40.878245 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:40.878250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:40.878449 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:40.878604 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:40.878997 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.880262 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.881863 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.882801 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.882967 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:40.883157 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:40.883343 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 02:46:40.883455 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":67,"threads_started":1}
I20260811 02:46:40.883631 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:40.877037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:37691 (local address 127.0.0.1:59988)
0811 02:46:40.877510 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:40.877555 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:40.877855 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:40.878625 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:40.878631 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:40.878652 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:40.878801 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:40.878808 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.880438 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.880450 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:40.881872 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:40.881883 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.883014 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.883019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:40.883150 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:40.883184 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 02:46:40.883293 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 02:46:40.883459 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":176,"spinlock_wait_cycles":5376,"thread_start_us":90,"threads_started":1}
I20260811 02:46:40.883975 24119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:40.878794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56270 (local address 127.21.236.130:33673)
0811 02:46:40.879265 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:40.879280 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:40.879307 (+ 27us) server_negotiation.cc:407] Connection header received
0811 02:46:40.879459 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:40.879464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:40.879619 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:40.879783 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:40.879998 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.881146 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.882433 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.883049 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.883187 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:40.883320 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:40.883718 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 02:46:40.883811 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":127,"threads_started":1}
I20260811 02:46:40.884461 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:40.878618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:33673 (local address 127.0.0.1:56270)
0811 02:46:40.879211 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:40.879226 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:40.879325 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:40.879759 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:40.879762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:40.879781 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:40.879889 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:40.879896 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.881303 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.881306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:40.882303 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:40.882311 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.883397 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.883402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:40.883486 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:40.883506 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 02:46:40.883580 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 02:46:40.883639 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":1,"thread_start_us":112,"threads_started":1}
I20260811 02:46:40.884568 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:40.877155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59988 (local address 127.21.236.131:37691)
0811 02:46:40.878168 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:40.878180 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:40.878200 (+ 20us) server_negotiation.cc:407] Connection header received
0811 02:46:40.878261 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:40.878265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:40.878449 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:40.878977 (+ 528us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:40.879176 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.880274 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.882012 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.882852 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.884082 (+ 1230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:40.884238 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:40.884295 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 02:46:40.884398 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":923,"thread_start_us":161,"threads_started":1}
I20260811 02:46:40.883651 24105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:40.877149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:34085 (local address 127.0.0.1:45066)
0811 02:46:40.877678 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:40.877692 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:40.877855 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:40.878759 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:40.878763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:40.878777 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:40.878873 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:40.878879 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.880399 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.880403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:40.881717 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:40.881727 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:40.882956 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:40.882960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:40.883107 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:40.883174 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 02:46:40.883288 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 02:46:40.883465 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":59,"spinlock_wait_cycles":8320,"thread_start_us":104,"threads_started":1}
I20260811 02:46:40.887390 23981 tablet_service.cc:1467] Processing CreateTablet for tablet 25ec60575b794f09ab47a08f73d9b7c2 (DEFAULT_TABLE table=test-table [id=f6e41d50b64f4edc90232294ae7d26a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:40.887356 23552 tablet_service.cc:1467] Processing CreateTablet for tablet 25ec60575b794f09ab47a08f73d9b7c2 (DEFAULT_TABLE table=test-table [id=f6e41d50b64f4edc90232294ae7d26a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:40.887382 23753 tablet_service.cc:1467] Processing CreateTablet for tablet 25ec60575b794f09ab47a08f73d9b7c2 (DEFAULT_TABLE table=test-table [id=f6e41d50b64f4edc90232294ae7d26a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:40.888522 23981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25ec60575b794f09ab47a08f73d9b7c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:40.888520 23753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25ec60575b794f09ab47a08f73d9b7c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:40.888526 23552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25ec60575b794f09ab47a08f73d9b7c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:40.895193 24120 tablet_bootstrap.cc:492] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff: Bootstrap starting.
I20260811 02:46:40.896566 24121 tablet_bootstrap.cc:492] T 25ec60575b794f09ab47a08f73d9b7c2 P 032189d726df457db4a44ef9c42a4d64: Bootstrap starting.
I20260811 02:46:40.896566 24123 tablet_bootstrap.cc:492] T 25ec60575b794f09ab47a08f73d9b7c2 P 1a6dc1debab84c0d91dc83c1b41c8575: Bootstrap starting.
I20260811 02:46:40.897631 24120 tablet_bootstrap.cc:654] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:40.898630 24120 log.cc:824] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:40.898778 24121 tablet_bootstrap.cc:654] T 25ec60575b794f09ab47a08f73d9b7c2 P 032189d726df457db4a44ef9c42a4d64: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:40.899355 24123 tablet_bootstrap.cc:654] T 25ec60575b794f09ab47a08f73d9b7c2 P 1a6dc1debab84c0d91dc83c1b41c8575: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:40.900108 24121 log.cc:824] T 25ec60575b794f09ab47a08f73d9b7c2 P 032189d726df457db4a44ef9c42a4d64: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:40.900241 24123 log.cc:824] T 25ec60575b794f09ab47a08f73d9b7c2 P 1a6dc1debab84c0d91dc83c1b41c8575: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:40.900876 24120 tablet_bootstrap.cc:492] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff: No bootstrap required, opened a new log
I20260811 02:46:40.901127 24120 ts_tablet_manager.cc:1397] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 02:46:40.902020 24121 tablet_bootstrap.cc:492] T 25ec60575b794f09ab47a08f73d9b7c2 P 032189d726df457db4a44ef9c42a4d64: No bootstrap required, opened a new log
I20260811 02:46:40.902271 24121 ts_tablet_manager.cc:1397] T 25ec60575b794f09ab47a08f73d9b7c2 P 032189d726df457db4a44ef9c42a4d64: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:46:40.904592 24123 tablet_bootstrap.cc:492] T 25ec60575b794f09ab47a08f73d9b7c2 P 1a6dc1debab84c0d91dc83c1b41c8575: No bootstrap required, opened a new log
I20260811 02:46:40.905474 24121 raft_consensus.cc:348] T 25ec60575b794f09ab47a08f73d9b7c2 P 032189d726df457db4a44ef9c42a4d64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032189d726df457db4a44ef9c42a4d64" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 37691 } } peers { permanent_uuid: "6bfb3efb6b094a989c8e245a35b467ff" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34085 } } peers { permanent_uuid: "1a6dc1debab84c0d91dc83c1b41c8575" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33673 } }
I20260811 02:46:40.905697 24121 raft_consensus.cc:374] T 25ec60575b794f09ab47a08f73d9b7c2 P 032189d726df457db4a44ef9c42a4d64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:40.905732 24121 raft_consensus.cc:729] T 25ec60575b794f09ab47a08f73d9b7c2 P 032189d726df457db4a44ef9c42a4d64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 032189d726df457db4a44ef9c42a4d64, State: Initialized, Role: FOLLOWER
I20260811 02:46:40.906000 24123 ts_tablet_manager.cc:1397] T 25ec60575b794f09ab47a08f73d9b7c2 P 1a6dc1debab84c0d91dc83c1b41c8575: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260811 02:46:40.906033 24120 raft_consensus.cc:348] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032189d726df457db4a44ef9c42a4d64" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 37691 } } peers { permanent_uuid: "6bfb3efb6b094a989c8e245a35b467ff" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34085 } } peers { permanent_uuid: "1a6dc1debab84c0d91dc83c1b41c8575" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33673 } }
I20260811 02:46:40.906204 24121 consensus_queue.cc:260] T 25ec60575b794f09ab47a08f73d9b7c2 P 032189d726df457db4a44ef9c42a4d64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032189d726df457db4a44ef9c42a4d64" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 37691 } } peers { permanent_uuid: "6bfb3efb6b094a989c8e245a35b467ff" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34085 } } peers { permanent_uuid: "1a6dc1debab84c0d91dc83c1b41c8575" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33673 } }
I20260811 02:46:40.906322 24120 raft_consensus.cc:374] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:40.906387 24120 raft_consensus.cc:729] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bfb3efb6b094a989c8e245a35b467ff, State: Initialized, Role: FOLLOWER
I20260811 02:46:40.906831 24120 consensus_queue.cc:260] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032189d726df457db4a44ef9c42a4d64" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 37691 } } peers { permanent_uuid: "6bfb3efb6b094a989c8e245a35b467ff" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34085 } } peers { permanent_uuid: "1a6dc1debab84c0d91dc83c1b41c8575" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33673 } }
I20260811 02:46:40.906975 24090 heartbeater.cc:503] Master 127.21.236.190:34047 was elected leader, sending a full tablet report...
I20260811 02:46:40.907320 24121 ts_tablet_manager.cc:1428] T 25ec60575b794f09ab47a08f73d9b7c2 P 032189d726df457db4a44ef9c42a4d64: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260811 02:46:40.907847 24120 ts_tablet_manager.cc:1428] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 02:46:40.908380 23653 heartbeater.cc:503] Master 127.21.236.190:34047 was elected leader, sending a full tablet report...
I20260811 02:46:40.909744 24123 raft_consensus.cc:348] T 25ec60575b794f09ab47a08f73d9b7c2 P 1a6dc1debab84c0d91dc83c1b41c8575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032189d726df457db4a44ef9c42a4d64" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 37691 } } peers { permanent_uuid: "6bfb3efb6b094a989c8e245a35b467ff" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34085 } } peers { permanent_uuid: "1a6dc1debab84c0d91dc83c1b41c8575" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33673 } }
I20260811 02:46:40.909957 24123 raft_consensus.cc:374] T 25ec60575b794f09ab47a08f73d9b7c2 P 1a6dc1debab84c0d91dc83c1b41c8575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:40.909994 24123 raft_consensus.cc:729] T 25ec60575b794f09ab47a08f73d9b7c2 P 1a6dc1debab84c0d91dc83c1b41c8575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a6dc1debab84c0d91dc83c1b41c8575, State: Initialized, Role: FOLLOWER
I20260811 02:46:40.910506 24123 consensus_queue.cc:260] T 25ec60575b794f09ab47a08f73d9b7c2 P 1a6dc1debab84c0d91dc83c1b41c8575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032189d726df457db4a44ef9c42a4d64" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 37691 } } peers { permanent_uuid: "6bfb3efb6b094a989c8e245a35b467ff" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34085 } } peers { permanent_uuid: "1a6dc1debab84c0d91dc83c1b41c8575" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33673 } }
I20260811 02:46:40.911206 23856 heartbeater.cc:503] Master 127.21.236.190:34047 was elected leader, sending a full tablet report...
I20260811 02:46:40.911522 24123 ts_tablet_manager.cc:1428] T 25ec60575b794f09ab47a08f73d9b7c2 P 1a6dc1debab84c0d91dc83c1b41c8575: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260811 02:46:41.079119 24093 tablet.cc:2369] T 25ec60575b794f09ab47a08f73d9b7c2 P 032189d726df457db4a44ef9c42a4d64: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:46:41.096933 23857 tablet.cc:2369] T 25ec60575b794f09ab47a08f73d9b7c2 P 1a6dc1debab84c0d91dc83c1b41c8575: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:46:41.103933 23654 tablet.cc:2369] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:46:41.119663 24132 raft_consensus.cc:482] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:46:41.119935 24132 raft_consensus.cc:504] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032189d726df457db4a44ef9c42a4d64" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 37691 } } peers { permanent_uuid: "6bfb3efb6b094a989c8e245a35b467ff" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34085 } } peers { permanent_uuid: "1a6dc1debab84c0d91dc83c1b41c8575" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33673 } }
I20260811 02:46:41.121178 24132 leader_election.cc:288] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 032189d726df457db4a44ef9c42a4d64 (127.21.236.131:37691), 1a6dc1debab84c0d91dc83c1b41c8575 (127.21.236.130:33673)
I20260811 02:46:41.127424 24119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:41.121705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:39539 (local address 127.21.236.130:33673)
0811 02:46:41.121844 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:41.121849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:41.122152 (+ 303us) server_negotiation.cc:407] Connection header received
0811 02:46:41.122334 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:41.122338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:41.122396 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:41.122495 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:41.123013 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:41.123829 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:41.126021 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:41.126642 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:41.126789 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:41.126856 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:41.127213 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 02:46:41.127286 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:46:41.127609 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:41.121547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:33673 (local address 127.21.236.129:39539)
0811 02:46:41.122001 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:41.122018 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:41.122180 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:41.122544 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:41.122547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:41.122576 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:41.122894 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:41.122901 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:41.123992 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:41.123996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:41.125861 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:41.125876 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:41.126797 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:41.126802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:41.127014 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:41.127091 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 02:46:41.127183 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 02:46:41.127363 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":510,"spinlock_wait_cycles":24192,"thread_start_us":97,"threads_started":1}
I20260811 02:46:41.127621 24135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:41.121547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:37691 (local address 127.21.236.129:37637)
0811 02:46:41.122001 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:41.122018 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:41.122180 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:41.122544 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:41.122547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:41.122576 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:41.122878 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:41.122888 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:41.123992 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:41.123996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:41.125861 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:41.125876 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:41.126797 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:41.126802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:41.127007 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:41.127092 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 02:46:41.127183 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 02:46:41.127363 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":249,"spinlock_wait_cycles":25600,"thread_start_us":134,"threads_started":1}
I20260811 02:46:41.127864 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:41.121732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:37637 (local address 127.21.236.131:37691)
0811 02:46:41.121859 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:41.121864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:41.122152 (+ 288us) server_negotiation.cc:407] Connection header received
0811 02:46:41.122334 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:41.122338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:41.122402 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:41.122493 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:41.123013 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:41.123833 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:41.126021 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:41.126655 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:41.126790 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:41.126844 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:41.127213 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 02:46:41.127286 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 02:46:41.128348 23790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25ec60575b794f09ab47a08f73d9b7c2" candidate_uuid: "6bfb3efb6b094a989c8e245a35b467ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a6dc1debab84c0d91dc83c1b41c8575" is_pre_election: true
I20260811 02:46:41.128453 24014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25ec60575b794f09ab47a08f73d9b7c2" candidate_uuid: "6bfb3efb6b094a989c8e245a35b467ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "032189d726df457db4a44ef9c42a4d64" is_pre_election: true
I20260811 02:46:41.128718 23790 raft_consensus.cc:2454] T 25ec60575b794f09ab47a08f73d9b7c2 P 1a6dc1debab84c0d91dc83c1b41c8575 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bfb3efb6b094a989c8e245a35b467ff in term 0.
I20260811 02:46:41.128814 24014 raft_consensus.cc:2454] T 25ec60575b794f09ab47a08f73d9b7c2 P 032189d726df457db4a44ef9c42a4d64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bfb3efb6b094a989c8e245a35b467ff in term 0.
I20260811 02:46:41.129248 23475 leader_election.cc:302] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a6dc1debab84c0d91dc83c1b41c8575, 6bfb3efb6b094a989c8e245a35b467ff; no voters:
I20260811 02:46:41.129550 24132 raft_consensus.cc:2781] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:46:41.129652 24132 raft_consensus.cc:482] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:46:41.129714 24132 raft_consensus.cc:3037] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:41.130779 24132 raft_consensus.cc:504] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032189d726df457db4a44ef9c42a4d64" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 37691 } } peers { permanent_uuid: "6bfb3efb6b094a989c8e245a35b467ff" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34085 } } peers { permanent_uuid: "1a6dc1debab84c0d91dc83c1b41c8575" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33673 } }
I20260811 02:46:41.131162 24132 leader_election.cc:288] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [CANDIDATE]: Term 1 election: Requested vote from peers 032189d726df457db4a44ef9c42a4d64 (127.21.236.131:37691), 1a6dc1debab84c0d91dc83c1b41c8575 (127.21.236.130:33673)
I20260811 02:46:41.131531 23790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25ec60575b794f09ab47a08f73d9b7c2" candidate_uuid: "6bfb3efb6b094a989c8e245a35b467ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a6dc1debab84c0d91dc83c1b41c8575"
I20260811 02:46:41.131563 24014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25ec60575b794f09ab47a08f73d9b7c2" candidate_uuid: "6bfb3efb6b094a989c8e245a35b467ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "032189d726df457db4a44ef9c42a4d64"
I20260811 02:46:41.131671 23790 raft_consensus.cc:3037] T 25ec60575b794f09ab47a08f73d9b7c2 P 1a6dc1debab84c0d91dc83c1b41c8575 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:41.131702 24014 raft_consensus.cc:3037] T 25ec60575b794f09ab47a08f73d9b7c2 P 032189d726df457db4a44ef9c42a4d64 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:41.132702 23790 raft_consensus.cc:2454] T 25ec60575b794f09ab47a08f73d9b7c2 P 1a6dc1debab84c0d91dc83c1b41c8575 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bfb3efb6b094a989c8e245a35b467ff in term 1.
I20260811 02:46:41.132794 24014 raft_consensus.cc:2454] T 25ec60575b794f09ab47a08f73d9b7c2 P 032189d726df457db4a44ef9c42a4d64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bfb3efb6b094a989c8e245a35b467ff in term 1.
I20260811 02:46:41.133046 23475 leader_election.cc:302] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a6dc1debab84c0d91dc83c1b41c8575, 6bfb3efb6b094a989c8e245a35b467ff; no voters:
I20260811 02:46:41.133255 24132 raft_consensus.cc:2781] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:41.133505 24132 raft_consensus.cc:686] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [term 1 LEADER]: Becoming Leader. State: Replica: 6bfb3efb6b094a989c8e245a35b467ff, State: Running, Role: LEADER
I20260811 02:46:41.133838 24132 consensus_queue.cc:237] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032189d726df457db4a44ef9c42a4d64" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 37691 } } peers { permanent_uuid: "6bfb3efb6b094a989c8e245a35b467ff" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34085 } } peers { permanent_uuid: "1a6dc1debab84c0d91dc83c1b41c8575" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33673 } }
I20260811 02:46:41.137238 23359 catalog_manager.cc:5495] T 25ec60575b794f09ab47a08f73d9b7c2 P 6bfb3efb6b094a989c8e245a35b467ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bfb3efb6b094a989c8e245a35b467ff (127.21.236.129). New cstate: current_term: 1 leader_uuid: "6bfb3efb6b094a989c8e245a35b467ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032189d726df457db4a44ef9c42a4d64" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 37691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6bfb3efb6b094a989c8e245a35b467ff" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a6dc1debab84c0d91dc83c1b41c8575" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33673 } health_report { overall_health: UNKNOWN } } }
Aug 11 02:46:41 dist-test-slave-n326 krb5kdc[23285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.129@KRBTEST.COM
I20260811 02:46:41.178427 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:41.168695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45076 (local address 127.21.236.129:34085)
0811 02:46:41.168906 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:41.168912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:41.169084 (+ 172us) server_negotiation.cc:407] Connection header received
0811 02:46:41.169256 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:41.169259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:41.169314 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:41.169405 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:41.170167 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:41.170710 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:41.171626 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:41.171852 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:41.171979 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:41.174548 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:41.174565 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:41.174582 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:41.174608 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:41.176672 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:41.177139 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:41.177147 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:41.177148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:41.177289 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:41.177552 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:41.177554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:41.177555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:41.177812 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:41.177983 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:41.178196 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 02:46:41.178300 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
Aug 11 02:46:41 dist-test-slave-n326 krb5kdc[23285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416401, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:46:41 dist-test-slave-n326 krb5kdc[23285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416401, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.236.129@KRBTEST.COM
I20260811 02:46:41.207492 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:41.198545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45078 (local address 127.21.236.129:34085)
0811 02:46:41.198740 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:41.198744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:41.198832 (+ 88us) server_negotiation.cc:407] Connection header received
0811 02:46:41.199081 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:41.199132 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:41.199233 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:41.199324 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:41.200026 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:41.200673 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:41.201454 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:41.201698 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:41.201775 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:41.204381 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:41.204384 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:41.204386 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:41.204411 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:41.205802 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:41.206252 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:41.206254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:41.206255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:41.206338 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:41.206594 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:41.206596 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:41.206597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:41.206760 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:41.206864 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:41.207238 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 02:46:41.207369 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 02:46:41 dist-test-slave-n326 krb5kdc[23285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:46:41 dist-test-slave-n326 krb5kdc[23285](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.129@KRBTEST.COM
I20260811 02:46:41.236307 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:41.227935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45092 (local address 127.21.236.129:34085)
0811 02:46:41.228104 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:41.228108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:41.228152 (+ 44us) server_negotiation.cc:407] Connection header received
0811 02:46:41.228296 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:41.228299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:41.228344 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:41.228420 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:41.229169 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:41.229685 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:41.230543 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:41.230773 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:41.230860 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:41.233449 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:41.233452 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:41.233454 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:41.233480 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:41.234767 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:41.235200 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:41.235203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:41.235204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:41.235281 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:41.235511 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:41.235514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:41.235515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:41.235695 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:41.235799 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:41.235997 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 02:46:41.236103 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 02:46:41.237812 23521 tablet_service.cc:2908] Rejecting scan request for tablet 25ec60575b794f09ab47a08f73d9b7c2: Uninitialized: clean time has not yet been initialized
I20260811 02:46:41.240077 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 23452
I20260811 02:46:41.260587 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 23661
I20260811 02:46:41.274835 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 23867
I20260811 02:46:41.290241 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 23310
2026-08-11T02:46:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2902 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:46:41 dist-test-slave-n326 krb5kdc[24163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:46:41 dist-test-slave-n326 krb5kdc[24163](info): set up 2 sockets
Aug 11 02:46:41 dist-test-slave-n326 krb5kdc[24163](info): commencing operation
krb5kdc: starting...
W20260811 02:46:42.610476 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:46:42 dist-test-slave-n326 krb5kdc[24163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:46:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:46:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:46:42.758846 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:43741
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:43741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:42.882303 24187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:42.882622 24187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:42.882665 24187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:42.886722 24187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:46:42.886790 24187 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 02:46:42.886807 24187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:42.886826 24187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:42.886842 24187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:46:42.886858 24187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:46:42.892283 24187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:43741
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:43741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:42.893404 24187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:42.894429 24187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:42.904063 24198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:42.904063 24195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:42.904105 24200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:42.904532 24187 server_base.cc:1034] running on GCE node
I20260811 02:46:42.905189 24187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:42.906638 24187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:42.907850 24187 hybrid_clock.cc:648] HybridClock initialized: now 1786416402907822 us; error 63 us; skew 500 ppm
Aug 11 02:46:42 dist-test-slave-n326 krb5kdc[24163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416402, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:42.911155 24187 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:46:42.912165 24187 webserver.cc:460] Webserver started at http://127.21.236.190:35015/ using document root <none> and password file <none>
I20260811 02:46:42.912796 24187 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:42.912873 24187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:42.913082 24187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:42.915479 24187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "00b264a7454542489aa2d99d53e3e060"
format_stamp: "Formatted at 2026-08-11 02:46:42 on dist-test-slave-n326"
server_key: "8421e85317fa79f1b61df98bee61b077"
server_key_iv: "08b2690aa28ce71bcb6d498abeff2547"
server_key_version: "encryptionkey@0"
I20260811 02:46:42.916082 24187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "00b264a7454542489aa2d99d53e3e060"
format_stamp: "Formatted at 2026-08-11 02:46:42 on dist-test-slave-n326"
server_key: "8421e85317fa79f1b61df98bee61b077"
server_key_iv: "08b2690aa28ce71bcb6d498abeff2547"
server_key_version: "encryptionkey@0"
I20260811 02:46:42.920420 24187 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 02:46:42.923178 24216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:42.924567 24187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:46:42.924682 24187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "00b264a7454542489aa2d99d53e3e060"
format_stamp: "Formatted at 2026-08-11 02:46:42 on dist-test-slave-n326"
server_key: "8421e85317fa79f1b61df98bee61b077"
server_key_iv: "08b2690aa28ce71bcb6d498abeff2547"
server_key_version: "encryptionkey@0"
I20260811 02:46:42.924791 24187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:42.965382 24187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:42.969094 24187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:42.969374 24187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:42.979017 24187 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:43741
I20260811 02:46:42.979036 24330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:43741 every 8 connection(s)
I20260811 02:46:42.980136 24187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:46:42.983109 24331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:42.988184 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 24187
I20260811 02:46:42.988284 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:46:42.988575 22450 external_mini_cluster.cc:1428] Setting key ae0bc2793dd053db9c37d3a1c44b9a5d
I20260811 02:46:42.988912 24331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060: Bootstrap starting.
I20260811 02:46:42.991627 24331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:42.992539 24331 log.cc:824] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:42.994951 24331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060: No bootstrap required, opened a new log
Aug 11 02:46:42 dist-test-slave-n326 krb5kdc[24163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:42.999327 24331 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b264a7454542489aa2d99d53e3e060" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43741 } }
I20260811 02:46:42.999647 24331 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:42.999720 24331 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b264a7454542489aa2d99d53e3e060, State: Initialized, Role: FOLLOWER
I20260811 02:46:43.000250 24331 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b264a7454542489aa2d99d53e3e060" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43741 } }
I20260811 02:46:43.000451 24331 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:46:43.000532 24331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:46:43.000656 24331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:43.001606 24331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b264a7454542489aa2d99d53e3e060" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43741 } }
I20260811 02:46:43.001956 24331 leader_election.cc:302] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00b264a7454542489aa2d99d53e3e060; no voters:
I20260811 02:46:43.002305 24331 leader_election.cc:288] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:46:43.002379 24338 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:43.002610 24338 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [term 1 LEADER]: Becoming Leader. State: Replica: 00b264a7454542489aa2d99d53e3e060, State: Running, Role: LEADER
I20260811 02:46:43.002915 24338 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b264a7454542489aa2d99d53e3e060" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43741 } }
I20260811 02:46:43.003710 24331 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:46:43.004153 24338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00b264a7454542489aa2d99d53e3e060. Latest consensus state: current_term: 1 leader_uuid: "00b264a7454542489aa2d99d53e3e060" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b264a7454542489aa2d99d53e3e060" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43741 } } }
I20260811 02:46:43.004259 24338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [sys.catalog]: This master's current role is: LEADER
I20260811 02:46:43.004448 24338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00b264a7454542489aa2d99d53e3e060" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00b264a7454542489aa2d99d53e3e060" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43741 } } }
I20260811 02:46:43.004550 24338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060 [sys.catalog]: This master's current role is: LEADER
I20260811 02:46:43.004793 24345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:46:43.005908 24336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:42.990134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37422 (local address 127.21.236.190:43741)
0811 02:46:42.990611 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:42.990630 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:42.990672 (+ 42us) server_negotiation.cc:407] Connection header received
0811 02:46:42.991548 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:42.991574 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:42.992029 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:42.992399 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:42.993336 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:42.994822 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:42.995798 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:42.996568 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:42.996760 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:42.999575 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:42.999582 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:42.999599 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:42.999634 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:43.001656 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:43.002245 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:43.002251 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:43.002292 (+ 41us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:43.002501 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:43.002885 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:43.002889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:43.002891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:43.003431 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:43.003685 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:43.004025 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 02:46:43.004382 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":138,"threads_started":1}
I20260811 02:46:43.009019 24345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:46:43.014777 24345 catalog_manager.cc:1347] Generated new cluster ID: 2eaff3718d654bedab6481ee6d25ffcc
I20260811 02:46:43.014858 24345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:46:43.026201 24345 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:46:43.027416 24345 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:46:43.042497 24345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060: Generated new TSK 0
I20260811 02:46:43.043404 24345 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:46:43.074777 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43741
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:43.198433 24379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:43.198822 24379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:43.198949 24379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:43.206959 24379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:43.207037 24379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:43.207104 24379 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:46:43.207139 24379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:46:43.213114 24379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.236.190:43741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:43.214690 24379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:43.216032 24379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:43.228706 24388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:43.228734 24392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:43.228749 24389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:43.229108 24379 server_base.cc:1034] running on GCE node
I20260811 02:46:43.229646 24379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:43.230370 24379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:43.231575 24379 hybrid_clock.cc:648] HybridClock initialized: now 1786416403231544 us; error 42 us; skew 500 ppm
Aug 11 02:46:43 dist-test-slave-n326 krb5kdc[24163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416403, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:43.234591 24379 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:46:43.235527 24379 webserver.cc:460] Webserver started at http://127.21.236.129:45683/ using document root <none> and password file <none>
I20260811 02:46:43.236135 24379 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:43.236245 24379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:43.236488 24379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:43.239281 24379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "5eeccdf0c4eb49b69c037c898fab72b7"
format_stamp: "Formatted at 2026-08-11 02:46:43 on dist-test-slave-n326"
server_key: "c9183084ef32ec4989d015ebd9d48639"
server_key_iv: "ee5fefcfc48e35e4fc86a2e98068998c"
server_key_version: "encryptionkey@0"
I20260811 02:46:43.239887 24379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "5eeccdf0c4eb49b69c037c898fab72b7"
format_stamp: "Formatted at 2026-08-11 02:46:43 on dist-test-slave-n326"
server_key: "c9183084ef32ec4989d015ebd9d48639"
server_key_iv: "ee5fefcfc48e35e4fc86a2e98068998c"
server_key_version: "encryptionkey@0"
I20260811 02:46:43.243974 24379 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:46:43.246610 24403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:43.247993 24379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:46:43.248135 24379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "5eeccdf0c4eb49b69c037c898fab72b7"
format_stamp: "Formatted at 2026-08-11 02:46:43 on dist-test-slave-n326"
server_key: "c9183084ef32ec4989d015ebd9d48639"
server_key_iv: "ee5fefcfc48e35e4fc86a2e98068998c"
server_key_version: "encryptionkey@0"
I20260811 02:46:43.248253 24379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:43.283466 24379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:43.287184 24379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:43.287412 24379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:43.289016 24379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:43.289093 24379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:43.289165 24379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:43.289214 24379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:46:43 dist-test-slave-n326 krb5kdc[24163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416403, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:43.301616 24379 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:34717
I20260811 02:46:43.301748 24591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:34717 every 8 connection(s)
I20260811 02:46:43.302591 24379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:46:43.303929 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 24379
I20260811 02:46:43.304080 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:46:43.304402 22450 external_mini_cluster.cc:1428] Setting key e3321aaec518c663a3fa3fc1f3feac13
I20260811 02:46:43.306192 24415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:43.290604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43741 (local address 127.21.236.129:47937)
0811 02:46:43.291145 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:43.291204 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:43.292454 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:43.293151 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:43.293158 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:43.293663 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:43.294196 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:43.294211 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:43.295202 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:43.295206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:43.296258 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:43.296269 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:43.296888 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:43.296893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:43.297157 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:43.297936 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:43.297962 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:43.300175 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:43.302554 (+ 2379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:43.302561 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:43.302578 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:43.302943 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:43.303375 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:43.303382 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:43.303386 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:43.303620 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:43.304054 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:43.304062 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:43.304412 (+ 350us) client_negotiation.cc:769] Sending connection context
0811 02:46:43.304689 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 02:46:43.305107 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":151,"threads_started":1}
I20260811 02:46:43.307144 24336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:43.290175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:47937 (local address 127.21.236.190:43741)
0811 02:46:43.290336 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:43.290342 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:43.291315 (+ 973us) server_negotiation.cc:407] Connection header received
0811 02:46:43.292750 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:43.292755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:43.292826 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:43.292939 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:43.294370 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:43.295019 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:43.296414 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:43.296699 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:43.296827 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:43.300399 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:43.300404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:43.300408 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:43.300445 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:43.302388 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:43.303088 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:43.303094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:43.303098 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:43.303199 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:43.303711 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:43.303713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:43.303714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:43.303912 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:43.304002 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:43.306647 (+ 2645us) server_negotiation.cc:299] Negotiation successful
0811 02:46:43.306833 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 02:46:43.308373 24592 heartbeater.cc:347] Connected to a master server at 127.21.236.190:43741
I20260811 02:46:43.308568 24592 heartbeater.cc:464] Registering TS with master...
I20260811 02:46:43.309300 24592 heartbeater.cc:511] Master 127.21.236.190:43741 requested a full tablet report, sending...
I20260811 02:46:43.311231 24254 ts_manager.cc:194] Registered new tserver with Master: 5eeccdf0c4eb49b69c037c898fab72b7 (127.21.236.129:34717)
I20260811 02:46:43.312968 24254 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:47937
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
I20260811 02:46:43.321225 24605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:43.311144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43741 (local address 127.21.236.129:40997)
0811 02:46:43.311464 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:43.311488 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:43.311570 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:43.314203 (+ 2633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:43.314206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:43.314480 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:43.314604 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:43.314612 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:43.315767 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:43.315772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:43.316351 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:43.316359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:43.317010 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:43.317013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:43.317103 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:43.317669 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:43.317681 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:43.318016 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:43.319765 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:43.319768 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:43.319770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:43.319978 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:43.320317 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:43.320319 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:43.320321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:43.320418 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:43.320759 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:43.320761 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:43.320840 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 02:46:43.320921 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 02:46:43.321060 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":130,"threads_started":1}
I20260811 02:46:43.321230 24336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:43.313725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:40997 (local address 127.21.236.190:43741)
0811 02:46:43.313940 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:43.313946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:43.313965 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:46:43.314011 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:43.314014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:43.314069 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:43.314176 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:43.314753 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:43.315622 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:43.316482 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:43.316876 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:43.316975 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:43.318203 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:43.318207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:43.318210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:43.318243 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:43.319635 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:43.320090 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:43.320092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:43.320093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:43.320219 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:43.320502 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:43.320504 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:43.320505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:43.320675 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:43.320753 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:43.321002 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 02:46:43.321110 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
W20260811 02:46:44.259487 22450 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.236.130: real 0.953s user 0.000s sys 0.005s
I20260811 02:46:44.260228 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43741
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:46:44.316099 24592 heartbeater.cc:503] Master 127.21.236.190:43741 was elected leader, sending a full tablet report...
W20260811 02:46:44.381428 24609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:44.381688 24609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:44.381735 24609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:44.385860 24609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:44.385933 24609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:44.385979 24609 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:46:44.386076 24609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:46:44.392736 24609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.236.190:43741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:44.393874 24609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:44.395005 24609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:44.405143 24622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:44.405169 24619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:44.405162 24620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:44.405225 24609 server_base.cc:1034] running on GCE node
I20260811 02:46:44.405766 24609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:44.406546 24609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:44.407733 24609 hybrid_clock.cc:648] HybridClock initialized: now 1786416404407660 us; error 99 us; skew 500 ppm
Aug 11 02:46:44 dist-test-slave-n326 krb5kdc[24163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416404, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:44.410914 24609 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:46:44.411926 24609 webserver.cc:460] Webserver started at http://127.21.236.130:43771/ using document root <none> and password file <none>
I20260811 02:46:44.412531 24609 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:44.412629 24609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:44.412878 24609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:44.415053 24609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "638ddde6c6404ba28bbfb76554283327"
format_stamp: "Formatted at 2026-08-11 02:46:44 on dist-test-slave-n326"
server_key: "ffcbf6d17236791b819671a1a9579866"
server_key_iv: "4bd51f2bda32afd9d561444beba42a84"
server_key_version: "encryptionkey@0"
I20260811 02:46:44.415691 24609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "638ddde6c6404ba28bbfb76554283327"
format_stamp: "Formatted at 2026-08-11 02:46:44 on dist-test-slave-n326"
server_key: "ffcbf6d17236791b819671a1a9579866"
server_key_iv: "4bd51f2bda32afd9d561444beba42a84"
server_key_version: "encryptionkey@0"
I20260811 02:46:44.419683 24609 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:46:44.422338 24637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:44.423780 24609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:46:44.423930 24609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "638ddde6c6404ba28bbfb76554283327"
format_stamp: "Formatted at 2026-08-11 02:46:44 on dist-test-slave-n326"
server_key: "ffcbf6d17236791b819671a1a9579866"
server_key_iv: "4bd51f2bda32afd9d561444beba42a84"
server_key_version: "encryptionkey@0"
I20260811 02:46:44.424073 24609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:44.456007 24609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:44.459880 24609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:44.460114 24609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:44.461774 24609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:44.461862 24609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:44.461939 24609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:44.461990 24609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:46:44 dist-test-slave-n326 krb5kdc[24163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416404, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:44.473634 24837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:37513 every 8 connection(s)
I20260811 02:46:44.473417 24609 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:37513
I20260811 02:46:44.474922 24609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:46:44.477810 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:44.463617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:50911 (local address 127.21.236.190:43741)
0811 02:46:44.463881 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:44.463884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:44.464286 (+ 402us) server_negotiation.cc:407] Connection header received
0811 02:46:44.465521 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:44.465525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:44.465594 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:44.465701 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:44.467564 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:44.468474 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:44.470382 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:44.470615 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:44.470696 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:44.474052 (+ 3356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:44.474055 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:44.474058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:44.474084 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:44.475949 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:44.476436 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:44.476439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:44.476440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:44.476527 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:44.476843 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:44.476845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:44.476846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:44.477028 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:44.477105 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:44.477571 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 02:46:44.477677 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":107,"threads_started":1}
I20260811 02:46:44.478636 24654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:44.463617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43741 (local address 127.21.236.130:50911)
0811 02:46:44.464134 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:44.464177 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:44.465269 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:44.465965 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:44.465976 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:44.466521 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:44.467364 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:44.467391 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:44.468609 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:44.468614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:44.470235 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:44.470245 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:44.470788 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:44.470795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:44.471027 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:44.471864 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:44.471886 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:44.473882 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:44.476072 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:44.476077 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:44.476088 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:44.476344 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:44.476613 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:44.476615 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:44.476617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:44.476748 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:44.477118 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:44.477123 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:44.477345 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 02:46:44.477493 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 02:46:44.477748 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":141,"threads_started":1}
I20260811 02:46:44.478850 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 24609
I20260811 02:46:44.479017 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:46:44.479372 22450 external_mini_cluster.cc:1428] Setting key d5e1dcfb581c5331abbc5b8b837db24c
I20260811 02:46:44.480005 24838 heartbeater.cc:347] Connected to a master server at 127.21.236.190:43741
I20260811 02:46:44.480160 24838 heartbeater.cc:464] Registering TS with master...
I20260811 02:46:44.480651 24838 heartbeater.cc:511] Master 127.21.236.190:43741 requested a full tablet report, sending...
I20260811 02:46:44.483716 24255 ts_manager.cc:194] Registered new tserver with Master: 638ddde6c6404ba28bbfb76554283327 (127.21.236.130:37513)
I20260811 02:46:44.484572 24255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:50911
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
I20260811 02:46:44.491647 24848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:44.483368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43741 (local address 127.21.236.130:51139)
0811 02:46:44.483708 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:44.483720 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:44.483834 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:44.484067 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:44.484070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:44.484299 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:44.484442 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:44.484453 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:44.485439 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:44.485443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:44.486045 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:44.486054 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:44.486638 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:44.486642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:44.486742 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:44.487302 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:44.487322 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:44.487862 (+ 540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:44.489702 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:44.489706 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:44.489708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:44.489965 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:44.490363 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:44.490368 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:44.490373 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:44.490483 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:44.491143 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:44.491149 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:44.491275 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 02:46:44.491368 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 02:46:44.491481 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":3,"thread_start_us":121,"threads_started":1}
I20260811 02:46:44.491667 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:44.483362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:51139 (local address 127.21.236.190:43741)
0811 02:46:44.483568 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:44.483572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:44.483793 (+ 221us) server_negotiation.cc:407] Connection header received
0811 02:46:44.483912 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:44.483915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:44.483964 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:44.484051 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:44.484588 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:44.485304 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:44.486173 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:44.486507 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:44.486621 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:44.488064 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:44.488067 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:44.488070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:44.488092 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:44.489583 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:44.490111 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:44.490114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:44.490115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:44.490204 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:44.490606 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:44.490613 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:44.490618 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:44.490978 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:44.491151 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:44.491428 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 02:46:44.491545 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
W20260811 02:46:44.983026 22450 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.236.131: real 0.502s user 0.000s sys 0.004s
I20260811 02:46:44.983652 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43741
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:45.100317 24850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:45.100582 24850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:45.100627 24850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:45.104852 24850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:45.104916 24850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:45.104959 24850 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:46:45.104991 24850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:46:45.110316 24850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.236.190:43741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:45.111346 24850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:45.112282 24850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:45.122102 24857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:45.122131 24858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:45.122167 24861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:45.122152 24850 server_base.cc:1034] running on GCE node
I20260811 02:46:45.122869 24850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:45.123575 24850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:45.124763 24850 hybrid_clock.cc:648] HybridClock initialized: now 1786416405124739 us; error 31 us; skew 500 ppm
Aug 11 02:46:45 dist-test-slave-n326 krb5kdc[24163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416405, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:45.133303 24850 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:46:45.134325 24850 webserver.cc:460] Webserver started at http://127.21.236.131:37299/ using document root <none> and password file <none>
I20260811 02:46:45.134922 24850 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:45.135000 24850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:45.135260 24850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:45.137593 24850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb"
format_stamp: "Formatted at 2026-08-11 02:46:45 on dist-test-slave-n326"
server_key: "182feb21a4563220fc8bfd02fa192184"
server_key_iv: "8afdf3b2e9447a6149b7a2abd3990a4e"
server_key_version: "encryptionkey@0"
I20260811 02:46:45.138163 24850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb"
format_stamp: "Formatted at 2026-08-11 02:46:45 on dist-test-slave-n326"
server_key: "182feb21a4563220fc8bfd02fa192184"
server_key_iv: "8afdf3b2e9447a6149b7a2abd3990a4e"
server_key_version: "encryptionkey@0"
I20260811 02:46:45.142558 24850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:46:45.145382 24870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:45.146922 24850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:46:45.147141 24850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb"
format_stamp: "Formatted at 2026-08-11 02:46:45 on dist-test-slave-n326"
server_key: "182feb21a4563220fc8bfd02fa192184"
server_key_iv: "8afdf3b2e9447a6149b7a2abd3990a4e"
server_key_version: "encryptionkey@0"
I20260811 02:46:45.147293 24850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:45.163975 24850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:45.167618 24850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:45.167856 24850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:45.169519 24850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:45.169597 24850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:45.169674 24850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:45.169719 24850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:45.180958 24850 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:33701
Aug 11 02:46:45 dist-test-slave-n326 krb5kdc[24163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416405, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:45.181437 25091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:33701 every 8 connection(s)
I20260811 02:46:45.182190 24850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:46:45.185910 24887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.170295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:41899 (local address 127.21.236.190:43741)
0811 02:46:45.170626 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.170631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.171988 (+ 1357us) server_negotiation.cc:407] Connection header received
0811 02:46:45.173415 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.173419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.173483 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.173568 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:45.175435 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.176227 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.178120 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.178350 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.178442 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.181953 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:45.181958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:45.181961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:45.181993 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:45.183727 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:45.184264 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:45.184267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:45.184269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:45.184367 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:45.184743 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:45.184746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:45.184748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:45.184974 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:45.185070 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.185627 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.185751 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":99,"threads_started":1}
I20260811 02:46:45.186687 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.171287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43741 (local address 127.21.236.131:41899)
0811 02:46:45.171781 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.171849 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.173147 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.173816 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.173826 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.174410 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:45.175246 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.175272 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.176376 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.176383 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.177958 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.177972 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.178520 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.178524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.178781 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.179601 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:45.179628 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:45.181741 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:45.183838 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:45.183842 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:45.183853 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:45.184123 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:45.184497 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:45.184499 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:45.184500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:45.184640 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:45.185102 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:45.185109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:45.185363 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.185534 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.185807 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":110,"threads_started":1}
I20260811 02:46:45.188141 25093 heartbeater.cc:347] Connected to a master server at 127.21.236.190:43741
I20260811 02:46:45.188311 25093 heartbeater.cc:464] Registering TS with master...
I20260811 02:46:45.188843 25093 heartbeater.cc:511] Master 127.21.236.190:43741 requested a full tablet report, sending...
I20260811 02:46:45.190119 24254 ts_manager.cc:194] Registered new tserver with Master: fc43b02226bc45b5a2ff5e22ccf1f2fb (127.21.236.131:33701)
I20260811 02:46:45.191151 24254 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:41899
I20260811 02:46:45.191685 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 24850
I20260811 02:46:45.191820 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:46:45.192118 22450 external_mini_cluster.cc:1428] Setting key 3205c10b8e7c180ad6a1d728d0330bae
I20260811 02:46:45.194258 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:46:45.200004 25102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.190375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43741 (local address 127.21.236.131:39971)
0811 02:46:45.190707 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.190724 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.190838 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.191117 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.191120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.191372 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:45.191516 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.191529 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.192572 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.192575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.193141 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.193149 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.193688 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.193691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.193786 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.195199 (+ 1413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:45.195219 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:45.195725 (+ 506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:45.198152 (+ 2427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:45.198154 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:45.198156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:45.198414 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:45.198774 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:45.198776 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:45.198778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:45.198905 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:45.199478 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:45.199481 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:45.199597 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.199691 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.199811 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":142,"threads_started":1}
I20260811 02:46:45.200268 24887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.190461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:39971 (local address 127.21.236.190:43741)
0811 02:46:45.190604 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.190608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.190806 (+ 198us) server_negotiation.cc:407] Connection header received
0811 02:46:45.190938 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.190940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.190977 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.191111 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:45.191707 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.192427 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.193262 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.193569 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.193702 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.195918 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:45.195922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:45.195924 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:45.195950 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:45.198005 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:45.198530 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:45.198537 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:45.198542 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:45.198643 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:45.199022 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:45.199054 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:45.199075 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:45.199369 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:45.199487 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.199790 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.199979 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 02:46:45 dist-test-slave-n326 krb5kdc[24163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416405, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:46:45 dist-test-slave-n326 krb5kdc[24163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416405, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:45.224685 24887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.215884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37438 (local address 127.21.236.190:43741)
0811 02:46:45.216051 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.216055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.216074 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:46:45.216192 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.216194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.216247 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.216338 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:45.217085 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.217643 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.218531 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.218757 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.218874 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.221499 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:45.221503 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:45.221505 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:45.221534 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:45.222844 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:45.223380 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:45.223384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:45.223388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:45.223477 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:45.223775 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:45.223778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:45.223780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:45.223978 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:45.224136 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.224380 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.224504 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 02:46:45.227792 24254 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:46:45.231040 24254 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:46:45.246924 25131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.240127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:34717 (local address 127.0.0.1:40532)
0811 02:46:45.241373 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.241389 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.241483 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.242508 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.242511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.242529 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:45.242625 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.242633 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.244164 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.244166 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.245368 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.245379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.246280 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.246284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.246417 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.246462 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.246588 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.246719 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1166,"mutex_wait_us":89,"thread_start_us":68,"threads_started":1}
I20260811 02:46:45.247044 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.240971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45150 (local address 127.21.236.130:37513)
0811 02:46:45.241354 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.241375 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.241873 (+ 498us) server_negotiation.cc:407] Connection header received
0811 02:46:45.242111 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.242120 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.242319 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.242415 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:45.242894 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.243679 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.245417 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.246201 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.246299 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.246431 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.246813 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.246910 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":110,"threads_started":1}
I20260811 02:46:45.247056 25133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.241058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34132 (local address 127.21.236.131:33701)
0811 02:46:45.241507 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.241523 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.241552 (+ 29us) server_negotiation.cc:407] Connection header received
0811 02:46:45.241638 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.241643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.241873 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.242478 (+ 605us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:45.242624 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.243983 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.245714 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.246464 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.246609 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.246726 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.246818 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.246907 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":115,"threads_started":1}
I20260811 02:46:45.247520 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.241581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40532 (local address 127.21.236.129:34717)
0811 02:46:45.242048 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.242058 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.242082 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:46:45.242157 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.242162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.242360 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.242507 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:45.242751 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.243996 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.245534 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.246143 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.246294 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.246454 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.247286 (+ 832us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.247403 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":96,"threads_started":1}
I20260811 02:46:45.246977 25126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.239983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:33701 (local address 127.0.0.1:34132)
0811 02:46:45.240424 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.240466 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.240578 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.242109 (+ 1531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.242117 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.242143 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:45.242292 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.242299 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.244113 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.244116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.245583 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.245593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.246593 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.246598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.246688 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.246710 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.246795 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.246856 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":269,"thread_start_us":137,"threads_started":1}
I20260811 02:46:45.247448 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.240746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:37513 (local address 127.0.0.1:45150)
0811 02:46:45.241781 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.241795 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.241956 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.242662 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.242668 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.242688 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:45.242790 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.242794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.243793 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.243797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.245281 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.245293 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.246539 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.246544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.246645 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.246668 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.247239 (+ 571us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.247328 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":72,"thread_start_us":103,"threads_started":1}
I20260811 02:46:45.250699 24959 tablet_service.cc:1467] Processing CreateTablet for tablet acd0b7dfda0944c99fc0619736dd44a7 (DEFAULT_TABLE table=test-table [id=2173b3f17b76418f82b7540375f311e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:45.250816 24483 tablet_service.cc:1467] Processing CreateTablet for tablet acd0b7dfda0944c99fc0619736dd44a7 (DEFAULT_TABLE table=test-table [id=2173b3f17b76418f82b7540375f311e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:45.251322 24721 tablet_service.cc:1467] Processing CreateTablet for tablet acd0b7dfda0944c99fc0619736dd44a7 (DEFAULT_TABLE table=test-table [id=2173b3f17b76418f82b7540375f311e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:45.252050 24483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acd0b7dfda0944c99fc0619736dd44a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:45.252158 24959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acd0b7dfda0944c99fc0619736dd44a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:45.252348 24721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acd0b7dfda0944c99fc0619736dd44a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:45.260124 25137 tablet_bootstrap.cc:492] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327: Bootstrap starting.
I20260811 02:46:45.260645 25138 tablet_bootstrap.cc:492] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7: Bootstrap starting.
I20260811 02:46:45.260969 25136 tablet_bootstrap.cc:492] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Bootstrap starting.
I20260811 02:46:45.262646 25138 tablet_bootstrap.cc:654] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:45.262694 25137 tablet_bootstrap.cc:654] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:45.263113 25136 tablet_bootstrap.cc:654] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:45.263585 25138 log.cc:824] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:45.263651 25137 log.cc:824] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:45.263969 25136 log.cc:824] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:45.265822 25136 tablet_bootstrap.cc:492] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb: No bootstrap required, opened a new log
I20260811 02:46:45.265829 25138 tablet_bootstrap.cc:492] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7: No bootstrap required, opened a new log
I20260811 02:46:45.266001 25137 tablet_bootstrap.cc:492] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327: No bootstrap required, opened a new log
I20260811 02:46:45.266031 25138 ts_tablet_manager.cc:1397] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:46:45.266041 25136 ts_tablet_manager.cc:1397] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260811 02:46:45.266163 25137 ts_tablet_manager.cc:1397] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:46:45.269368 25136 raft_consensus.cc:348] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } }
I20260811 02:46:45.269641 25136 raft_consensus.cc:374] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:45.269721 25136 raft_consensus.cc:729] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc43b02226bc45b5a2ff5e22ccf1f2fb, State: Initialized, Role: FOLLOWER
I20260811 02:46:45.270241 25136 consensus_queue.cc:260] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } }
I20260811 02:46:45.270992 25093 heartbeater.cc:503] Master 127.21.236.190:43741 was elected leader, sending a full tablet report...
I20260811 02:46:45.271152 25137 raft_consensus.cc:348] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } }
I20260811 02:46:45.271152 25138 raft_consensus.cc:348] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } }
I20260811 02:46:45.271418 25137 raft_consensus.cc:374] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:45.271425 25136 ts_tablet_manager.cc:1428] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260811 02:46:45.271430 25138 raft_consensus.cc:374] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:45.271482 25138 raft_consensus.cc:729] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eeccdf0c4eb49b69c037c898fab72b7, State: Initialized, Role: FOLLOWER
I20260811 02:46:45.271481 25137 raft_consensus.cc:729] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 638ddde6c6404ba28bbfb76554283327, State: Initialized, Role: FOLLOWER
I20260811 02:46:45.271929 25137 consensus_queue.cc:260] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } }
I20260811 02:46:45.271929 25138 consensus_queue.cc:260] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } }
I20260811 02:46:45.272953 25138 ts_tablet_manager.cc:1428] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 02:46:45.272957 25137 ts_tablet_manager.cc:1428] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 02:46:45.273808 24838 heartbeater.cc:503] Master 127.21.236.190:43741 was elected leader, sending a full tablet report...
I20260811 02:46:45.302819 25146 raft_consensus.cc:482] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:46:45.303119 25146 raft_consensus.cc:504] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } }
I20260811 02:46:45.304719 25146 leader_election.cc:288] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc43b02226bc45b5a2ff5e22ccf1f2fb (127.21.236.131:33701), 5eeccdf0c4eb49b69c037c898fab72b7 (127.21.236.129:34717)
W20260811 02:46:45.305410 24594 tablet.cc:2369] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:46:45.310220 25133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.305603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:55995 (local address 127.21.236.131:33701)
0811 02:46:45.305773 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.305777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.305792 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:46:45.305843 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.305845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.305896 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.305988 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:45.306475 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.307254 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.308883 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.309478 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.309619 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.309671 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.310013 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.310094 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 02:46:45.310220 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.305450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:38505 (local address 127.21.236.129:34717)
0811 02:46:45.305559 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.305564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.306353 (+ 789us) server_negotiation.cc:407] Connection header received
0811 02:46:45.306559 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.306562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.306612 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.306690 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:45.306977 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.307602 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.308820 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.309488 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.309618 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.309671 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.310021 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.310106 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 02:46:45.310267 25150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.304959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:33701 (local address 127.21.236.130:55995)
0811 02:46:45.305461 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.305478 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.305591 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.306134 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.306136 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.306160 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:45.306285 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.306290 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.307358 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.307361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.308708 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.308719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.309643 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.309647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.309808 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.309886 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.309974 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.310070 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":73,"thread_start_us":162,"threads_started":1}
I20260811 02:46:45.310267 25148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.305025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:34717 (local address 127.21.236.130:38505)
0811 02:46:45.306268 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.306281 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.306417 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.306715 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.306718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.306734 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:45.306851 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.306856 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.307716 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.307720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.308684 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.308713 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.309643 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.309647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.309828 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.309887 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.309995 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.310074 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1108,"mutex_wait_us":113,"thread_start_us":116,"threads_started":1}
I20260811 02:46:45.311033 24993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd0b7dfda0944c99fc0619736dd44a7" candidate_uuid: "638ddde6c6404ba28bbfb76554283327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" is_pre_election: true
I20260811 02:46:45.311182 24512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd0b7dfda0944c99fc0619736dd44a7" candidate_uuid: "638ddde6c6404ba28bbfb76554283327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" is_pre_election: true
I20260811 02:46:45.311429 24993 raft_consensus.cc:2454] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 638ddde6c6404ba28bbfb76554283327 in term 0.
I20260811 02:46:45.311482 24512 raft_consensus.cc:2454] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 638ddde6c6404ba28bbfb76554283327 in term 0.
I20260811 02:46:45.311952 24638 leader_election.cc:302] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 638ddde6c6404ba28bbfb76554283327, fc43b02226bc45b5a2ff5e22ccf1f2fb; no voters:
I20260811 02:46:45.312251 25146 raft_consensus.cc:2781] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:46:45.312343 25146 raft_consensus.cc:482] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:46:45.312387 25146 raft_consensus.cc:3037] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:45.313524 25146 raft_consensus.cc:504] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } }
I20260811 02:46:45.313920 25146 leader_election.cc:288] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [CANDIDATE]: Term 1 election: Requested vote from peers fc43b02226bc45b5a2ff5e22ccf1f2fb (127.21.236.131:33701), 5eeccdf0c4eb49b69c037c898fab72b7 (127.21.236.129:34717)
I20260811 02:46:45.314256 24993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd0b7dfda0944c99fc0619736dd44a7" candidate_uuid: "638ddde6c6404ba28bbfb76554283327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb"
I20260811 02:46:45.314370 24512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acd0b7dfda0944c99fc0619736dd44a7" candidate_uuid: "638ddde6c6404ba28bbfb76554283327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eeccdf0c4eb49b69c037c898fab72b7"
I20260811 02:46:45.314409 24993 raft_consensus.cc:3037] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:45.314492 24512 raft_consensus.cc:3037] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:45.315783 24512 raft_consensus.cc:2454] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 638ddde6c6404ba28bbfb76554283327 in term 1.
I20260811 02:46:45.316123 24993 raft_consensus.cc:2454] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 638ddde6c6404ba28bbfb76554283327 in term 1.
I20260811 02:46:45.316128 24640 leader_election.cc:302] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5eeccdf0c4eb49b69c037c898fab72b7, 638ddde6c6404ba28bbfb76554283327; no voters:
I20260811 02:46:45.316309 25146 raft_consensus.cc:2781] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:45.316630 25146 raft_consensus.cc:686] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [term 1 LEADER]: Becoming Leader. State: Replica: 638ddde6c6404ba28bbfb76554283327, State: Running, Role: LEADER
I20260811 02:46:45.317003 25146 consensus_queue.cc:237] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } }
I20260811 02:46:45.320560 24254 catalog_manager.cc:5495] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 638ddde6c6404ba28bbfb76554283327 (127.21.236.130). New cstate: current_term: 1 leader_uuid: "638ddde6c6404ba28bbfb76554283327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } health_report { overall_health: HEALTHY } } }
I20260811 02:46:45.356369 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.352876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45158 (local address 127.21.236.130:37513)
0811 02:46:45.353036 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.353040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.353055 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:46:45.353219 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.353222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.353278 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.353370 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:46:45.353613 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.354111 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.354999 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.355242 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.355330 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.355618 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:46:45.356037 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:46:45.356125 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.356200 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.356255 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 02:46:45.362408 24512 raft_consensus.cc:1261] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 1 FOLLOWER]: Refusing update from remote peer 638ddde6c6404ba28bbfb76554283327: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:45.362408 24993 raft_consensus.cc:1261] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 1 FOLLOWER]: Refusing update from remote peer 638ddde6c6404ba28bbfb76554283327: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:45.363211 25152 consensus_queue.cc:1035] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:46:45.363404 25146 consensus_queue.cc:1035] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:46:45.372733 25164 mvcc.cc:205] Tried to move back new op lower bound from 7317161596356980736 to 7317161596181512192. Current Snapshot: MvccSnapshot[applied={T|T < 7317161596356980736}]
I20260811 02:46:45.384232 24253 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:46:45.384541 24253 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:46:45.387743 24253 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060: Sending DeleteTablet for 3 replicas of tablet acd0b7dfda0944c99fc0619736dd44a7
I20260811 02:46:45.388738 24959 tablet_service.cc:1514] Processing DeleteTablet for tablet acd0b7dfda0944c99fc0619736dd44a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:45 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:34132
I20260811 02:46:45.388904 24483 tablet_service.cc:1514] Processing DeleteTablet for tablet acd0b7dfda0944c99fc0619736dd44a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:45 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:40532
I20260811 02:46:45.389125 24721 tablet_service.cc:1514] Processing DeleteTablet for tablet acd0b7dfda0944c99fc0619736dd44a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:45 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:45150
I20260811 02:46:45.389323 25175 tablet_replica.cc:330] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb: stopping tablet replica
I20260811 02:46:45.389670 25175 raft_consensus.cc:2229] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:46:45.390118 25175 raft_consensus.cc:2258] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:45.390077 24253 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:46:45.391009 25177 tablet_replica.cc:330] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7: stopping tablet replica
W20260811 02:46:45.391323 24253 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:46:45.391350 25177 raft_consensus.cc:2229] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:46:45.391651 25177 raft_consensus.cc:2258] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:45.391897 25175 ts_tablet_manager.cc:1905] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:45.393059 25177 ts_tablet_manager.cc:1905] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:45.394801 25175 ts_tablet_manager.cc:1918] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:46:45.394902 25175 log.cc:1196] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/acd0b7dfda0944c99fc0619736dd44a7
I20260811 02:46:45.395301 25175 ts_tablet_manager.cc:1939] T acd0b7dfda0944c99fc0619736dd44a7 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Deleting consensus metadata
I20260811 02:46:45.395797 25177 ts_tablet_manager.cc:1918] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:46:45.395895 25177 log.cc:1196] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/wal/wals/acd0b7dfda0944c99fc0619736dd44a7
I20260811 02:46:45.396214 25177 ts_tablet_manager.cc:1939] T acd0b7dfda0944c99fc0619736dd44a7 P 5eeccdf0c4eb49b69c037c898fab72b7: Deleting consensus metadata
I20260811 02:46:45.396735 24217 catalog_manager.cc:4841] TS fc43b02226bc45b5a2ff5e22ccf1f2fb (127.21.236.131:33701): tablet acd0b7dfda0944c99fc0619736dd44a7 (table test-table [id=2173b3f17b76418f82b7540375f311e9]) successfully deleted
I20260811 02:46:45.397356 24222 catalog_manager.cc:4841] TS 5eeccdf0c4eb49b69c037c898fab72b7 (127.21.236.129:34717): tablet acd0b7dfda0944c99fc0619736dd44a7 (table test-table [id=2173b3f17b76418f82b7540375f311e9]) successfully deleted
I20260811 02:46:45.397599 25181 tablet_replica.cc:330] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327: stopping tablet replica
I20260811 02:46:45.397634 24483 tablet_service.cc:1467] Processing CreateTablet for tablet 8ae303b9f79647f1bbc5240b94b2b110 (DEFAULT_TABLE table=test-table [id=5bddf1929d444a94a02c7b9c2f9c3d56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:45.397891 24483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ae303b9f79647f1bbc5240b94b2b110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:45.397897 24959 tablet_service.cc:1467] Processing CreateTablet for tablet 8ae303b9f79647f1bbc5240b94b2b110 (DEFAULT_TABLE table=test-table [id=5bddf1929d444a94a02c7b9c2f9c3d56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:45.397966 25181 raft_consensus.cc:2229] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:46:45.398152 24959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ae303b9f79647f1bbc5240b94b2b110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:45.398386 25181 raft_consensus.cc:2258] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:45.399932 25181 ts_tablet_manager.cc:1905] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:45.400199 25136 tablet_bootstrap.cc:492] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Bootstrap starting.
I20260811 02:46:45.400314 25138 tablet_bootstrap.cc:492] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7: Bootstrap starting.
I20260811 02:46:45.400425 24721 tablet_service.cc:1467] Processing CreateTablet for tablet 8ae303b9f79647f1bbc5240b94b2b110 (DEFAULT_TABLE table=test-table [id=5bddf1929d444a94a02c7b9c2f9c3d56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:45.400678 24721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ae303b9f79647f1bbc5240b94b2b110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:45.401197 25136 tablet_bootstrap.cc:654] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:45.401306 25138 tablet_bootstrap.cc:654] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:45.402940 25181 ts_tablet_manager.cc:1918] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:46:45.403050 25181 log.cc:1196] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/acd0b7dfda0944c99fc0619736dd44a7
I20260811 02:46:45.403447 25181 ts_tablet_manager.cc:1939] T acd0b7dfda0944c99fc0619736dd44a7 P 638ddde6c6404ba28bbfb76554283327: Deleting consensus metadata
I20260811 02:46:45.403585 25137 tablet_bootstrap.cc:492] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327: Bootstrap starting.
I20260811 02:46:45.403869 25136 tablet_bootstrap.cc:492] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb: No bootstrap required, opened a new log
I20260811 02:46:45.403945 25136 ts_tablet_manager.cc:1397] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:46:45.404508 25136 raft_consensus.cc:348] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.404651 25136 raft_consensus.cc:374] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:45.404695 25136 raft_consensus.cc:729] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc43b02226bc45b5a2ff5e22ccf1f2fb, State: Initialized, Role: FOLLOWER
I20260811 02:46:45.404808 25137 tablet_bootstrap.cc:654] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:45.404808 24222 catalog_manager.cc:4841] TS 638ddde6c6404ba28bbfb76554283327 (127.21.236.130:37513): tablet acd0b7dfda0944c99fc0619736dd44a7 (table test-table [id=2173b3f17b76418f82b7540375f311e9]) successfully deleted
I20260811 02:46:45.404848 25136 consensus_queue.cc:260] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.405124 25136 ts_tablet_manager.cc:1428] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:46:45.405685 25138 tablet_bootstrap.cc:492] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7: No bootstrap required, opened a new log
I20260811 02:46:45.405777 25138 ts_tablet_manager.cc:1397] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 02:46:45.406052 25137 tablet_bootstrap.cc:492] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327: No bootstrap required, opened a new log
I20260811 02:46:45.406141 25137 ts_tablet_manager.cc:1397] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:46:45.406225 25138 raft_consensus.cc:348] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.406332 25138 raft_consensus.cc:374] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:45.406361 25138 raft_consensus.cc:729] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eeccdf0c4eb49b69c037c898fab72b7, State: Initialized, Role: FOLLOWER
I20260811 02:46:45.406488 25138 consensus_queue.cc:260] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.406607 25137 raft_consensus.cc:348] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.406723 25137 raft_consensus.cc:374] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:45.406733 25138 ts_tablet_manager.cc:1428] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:46:45.406760 25137 raft_consensus.cc:729] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 638ddde6c6404ba28bbfb76554283327, State: Initialized, Role: FOLLOWER
I20260811 02:46:45.406883 25137 consensus_queue.cc:260] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.407157 25137 ts_tablet_manager.cc:1428] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260811 02:46:45.433287 25096 tablet.cc:2369] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:46:45.477252 24839 tablet.cc:2369] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:46:45.482153 25144 raft_consensus.cc:482] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:46:45.482313 25144 raft_consensus.cc:504] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.483544 25144 leader_election.cc:288] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5eeccdf0c4eb49b69c037c898fab72b7 (127.21.236.129:34717), 638ddde6c6404ba28bbfb76554283327 (127.21.236.130:37513)
I20260811 02:46:45.489842 24889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.483957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:34717 (local address 127.21.236.131:60219)
0811 02:46:45.484194 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.484211 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.484307 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.485308 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.485311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.485330 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:45.485460 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.485467 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.486642 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.486646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.488107 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.488118 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.489246 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.489250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.489489 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.489540 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.489615 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.489692 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":87}
I20260811 02:46:45.489849 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.484904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:35287 (local address 127.21.236.130:37513)
0811 02:46:45.485085 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.485089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.485102 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:46:45.485147 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.485149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.485190 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.485254 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:45.486019 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.486759 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.488345 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.488981 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.489082 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.489141 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.489640 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.489724 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 02:46:45.490410 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.484579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:60219 (local address 127.21.236.129:34717)
0811 02:46:45.484710 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.484715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.484728 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:46:45.484783 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.484786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.484831 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.484903 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:45.485608 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.486497 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.488292 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.489006 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.489126 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.489180 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.490190 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.490283 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:46:45.490676 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.484199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:37513 (local address 127.21.236.131:35287)
0811 02:46:45.484609 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.484624 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.484723 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.485681 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.485684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.485706 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:45.485860 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.485867 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.486929 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.486934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.488195 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.488207 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.489276 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.489280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.489469 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.489514 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.490301 (+ 787us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.490385 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":40,"thread_start_us":76,"threads_started":1}
I20260811 02:46:45.490815 24512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae303b9f79647f1bbc5240b94b2b110" candidate_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" is_pre_election: true
I20260811 02:46:45.490989 24512 raft_consensus.cc:2454] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc43b02226bc45b5a2ff5e22ccf1f2fb in term 0.
I20260811 02:46:45.491520 24874 leader_election.cc:302] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5eeccdf0c4eb49b69c037c898fab72b7, fc43b02226bc45b5a2ff5e22ccf1f2fb; no voters:
I20260811 02:46:45.491678 24753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae303b9f79647f1bbc5240b94b2b110" candidate_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "638ddde6c6404ba28bbfb76554283327" is_pre_election: true
I20260811 02:46:45.491825 25144 raft_consensus.cc:2781] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:46:45.491885 25144 raft_consensus.cc:482] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:46:45.491946 25144 raft_consensus.cc:3037] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:45.491951 24753 raft_consensus.cc:2454] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc43b02226bc45b5a2ff5e22ccf1f2fb in term 0.
I20260811 02:46:45.493320 25144 raft_consensus.cc:504] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.493707 25144 leader_election.cc:288] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [CANDIDATE]: Term 1 election: Requested vote from peers 5eeccdf0c4eb49b69c037c898fab72b7 (127.21.236.129:34717), 638ddde6c6404ba28bbfb76554283327 (127.21.236.130:37513)
I20260811 02:46:45.494061 24512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae303b9f79647f1bbc5240b94b2b110" candidate_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5eeccdf0c4eb49b69c037c898fab72b7"
I20260811 02:46:45.494200 24512 raft_consensus.cc:3037] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:45.494203 24753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ae303b9f79647f1bbc5240b94b2b110" candidate_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "638ddde6c6404ba28bbfb76554283327"
I20260811 02:46:45.494323 24753 raft_consensus.cc:3037] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:45.495571 24512 raft_consensus.cc:2454] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc43b02226bc45b5a2ff5e22ccf1f2fb in term 1.
I20260811 02:46:45.495764 24753 raft_consensus.cc:2454] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc43b02226bc45b5a2ff5e22ccf1f2fb in term 1.
I20260811 02:46:45.495972 24874 leader_election.cc:302] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5eeccdf0c4eb49b69c037c898fab72b7, fc43b02226bc45b5a2ff5e22ccf1f2fb; no voters:
I20260811 02:46:45.496173 25144 raft_consensus.cc:2781] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:45.496470 25144 raft_consensus.cc:686] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 1 LEADER]: Becoming Leader. State: Replica: fc43b02226bc45b5a2ff5e22ccf1f2fb, State: Running, Role: LEADER
I20260811 02:46:45.496757 25144 consensus_queue.cc:237] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.499389 24255 catalog_manager.cc:5495] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb reported cstate change: term changed from 0 to 1, leader changed from <none> to fc43b02226bc45b5a2ff5e22ccf1f2fb (127.21.236.131). New cstate: current_term: 1 leader_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } health_report { overall_health: HEALTHY } } }
Aug 11 02:46:45 dist-test-slave-n326 krb5kdc[24163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416402, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:45.539706 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.529513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43741 (local address 127.0.0.1:37450)
0811 02:46:45.529902 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.529925 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.530035 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.530372 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.530375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.530815 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:45.530937 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.530944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.532076 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.532079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.532663 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.532673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.533284 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.533288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.533360 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.534061 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:45.534075 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:45.535834 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:45.537794 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:45.537804 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:45.537810 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:45.538149 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:45.538535 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:45.538538 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:45.538539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:45.538638 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:45.539195 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:45.539201 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:45.539310 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.539427 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.539555 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":174,"threads_started":1}
I20260811 02:46:45.539880 24887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.529513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37450 (local address 127.21.236.190:43741)
0811 02:46:45.529671 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.529677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.530011 (+ 334us) server_negotiation.cc:407] Connection header received
0811 02:46:45.530144 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.530148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.530204 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.530317 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:45.531106 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.531937 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.532800 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.533144 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.533251 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.535991 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:45.535995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:45.535998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:45.536024 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:45.537674 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:45.538324 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:45.538328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:45.538330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:45.538429 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:45.538736 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:45.538739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:45.538740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:45.538960 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:45.539091 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.539533 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.539691 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:46:45.544500 24255 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:37450:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 02:46:45.545445 24255 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:46:45.550946 24483 tablet_service.cc:1467] Processing CreateTablet for tablet 6aaeb76c68ef424f983998b349501447 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f3ac6415ce5f4b4c83a6e068228ddaa1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:46:45.550966 24721 tablet_service.cc:1467] Processing CreateTablet for tablet 6aaeb76c68ef424f983998b349501447 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f3ac6415ce5f4b4c83a6e068228ddaa1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:46:45.551177 24959 tablet_service.cc:1467] Processing CreateTablet for tablet 6aaeb76c68ef424f983998b349501447 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f3ac6415ce5f4b4c83a6e068228ddaa1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:46:45.551427 24721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aaeb76c68ef424f983998b349501447. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:45.551427 24483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aaeb76c68ef424f983998b349501447. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:45.551627 24959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aaeb76c68ef424f983998b349501447. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:45.554368 25137 tablet_bootstrap.cc:492] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327: Bootstrap starting.
I20260811 02:46:45.554564 25136 tablet_bootstrap.cc:492] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Bootstrap starting.
I20260811 02:46:45.555037 25137 tablet_bootstrap.cc:654] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:45.555490 25136 tablet_bootstrap.cc:654] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:45.556527 25138 tablet_bootstrap.cc:492] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7: Bootstrap starting.
I20260811 02:46:45.557142 25137 tablet_bootstrap.cc:492] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327: No bootstrap required, opened a new log
I20260811 02:46:45.557233 25137 ts_tablet_manager.cc:1397] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 02:46:45.557490 25138 tablet_bootstrap.cc:654] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:45.557673 25137 raft_consensus.cc:348] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.557765 25137 raft_consensus.cc:374] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:45.557785 25137 raft_consensus.cc:729] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 638ddde6c6404ba28bbfb76554283327, State: Initialized, Role: FOLLOWER
I20260811 02:46:45.557924 25137 consensus_queue.cc:260] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.558238 25137 ts_tablet_manager.cc:1428] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 02:46:45.558578 25136 tablet_bootstrap.cc:492] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb: No bootstrap required, opened a new log
I20260811 02:46:45.558667 25136 ts_tablet_manager.cc:1397] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:46:45.558611 25169 tablet_replica.cc:439] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } }
I20260811 02:46:45.558775 25169 tablet_replica.cc:442] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:46:45.559118 25136 raft_consensus.cc:348] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.559234 25136 raft_consensus.cc:374] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:45.559263 25136 raft_consensus.cc:729] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc43b02226bc45b5a2ff5e22ccf1f2fb, State: Initialized, Role: FOLLOWER
I20260811 02:46:45.559759 25138 tablet_bootstrap.cc:492] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7: No bootstrap required, opened a new log
I20260811 02:46:45.559847 25138 ts_tablet_manager.cc:1397] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:46:45.560274 25138 raft_consensus.cc:348] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.560388 25138 raft_consensus.cc:374] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:45.560428 25138 raft_consensus.cc:729] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5eeccdf0c4eb49b69c037c898fab72b7, State: Initialized, Role: FOLLOWER
I20260811 02:46:45.560590 25138 consensus_queue.cc:260] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.560896 25138 ts_tablet_manager.cc:1428] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:46:45.560917 25145 tablet_replica.cc:439] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } }
I20260811 02:46:45.561058 25145 tablet_replica.cc:442] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:46:45.561071 25136 consensus_queue.cc:260] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.561594 25144 tablet_replica.cc:439] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } }
I20260811 02:46:45.561748 25144 tablet_replica.cc:442] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:46:45.562057 25136 ts_tablet_manager.cc:1428] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260811 02:46:45.730425 25145 raft_consensus.cc:482] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:46:45.730597 25145 raft_consensus.cc:504] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.731931 25145 leader_election.cc:288] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 638ddde6c6404ba28bbfb76554283327 (127.21.236.130:37513), fc43b02226bc45b5a2ff5e22ccf1f2fb (127.21.236.131:33701)
I20260811 02:46:45.736361 25133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.732449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:57729 (local address 127.21.236.131:33701)
0811 02:46:45.732578 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.732583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.732976 (+ 393us) server_negotiation.cc:407] Connection header received
0811 02:46:45.733072 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.733075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.733118 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.733189 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:45.733496 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.734012 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.735269 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.735694 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.735830 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.735899 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.736126 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.736208 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:46:45.736380 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.732343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:57299 (local address 127.21.236.130:37513)
0811 02:46:45.732504 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.732510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.732769 (+ 259us) server_negotiation.cc:407] Connection header received
0811 02:46:45.732911 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.732916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.732977 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.733057 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:45.733560 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.734089 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.735304 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.735724 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.735859 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.735927 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.736154 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.736248 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 02:46:45.736404 25204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.732220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:33701 (local address 127.21.236.129:57729)
0811 02:46:45.732838 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.732854 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.732979 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.733230 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.733233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.733249 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:45.733358 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.733367 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.734130 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.734135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.735137 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.735147 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.735830 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.735834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.735987 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.736007 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.736103 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.736208 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":197,"thread_start_us":87,"threads_started":1}
I20260811 02:46:45.736516 25205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.732220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:37513 (local address 127.21.236.129:57299)
0811 02:46:45.732659 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.732672 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.732784 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.733286 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.733288 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.733304 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:45.733424 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.733429 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.734212 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.734218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.735173 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.735184 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.735852 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.735855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.736021 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.736040 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.736133 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.736242 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":73,"thread_start_us":122,"threads_started":1}
I20260811 02:46:45.737440 24753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aaeb76c68ef424f983998b349501447" candidate_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "638ddde6c6404ba28bbfb76554283327" is_pre_election: true
I20260811 02:46:45.737488 24993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aaeb76c68ef424f983998b349501447" candidate_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" is_pre_election: true
I20260811 02:46:45.737648 24753 raft_consensus.cc:2454] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5eeccdf0c4eb49b69c037c898fab72b7 in term 0.
I20260811 02:46:45.737654 24993 raft_consensus.cc:2454] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5eeccdf0c4eb49b69c037c898fab72b7 in term 0.
I20260811 02:46:45.738092 24407 leader_election.cc:302] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5eeccdf0c4eb49b69c037c898fab72b7, 638ddde6c6404ba28bbfb76554283327, fc43b02226bc45b5a2ff5e22ccf1f2fb; no voters:
I20260811 02:46:45.738420 25145 raft_consensus.cc:2781] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:46:45.738507 25145 raft_consensus.cc:482] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:46:45.738557 25145 raft_consensus.cc:3037] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:45.739673 25145 raft_consensus.cc:504] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.740140 25145 leader_election.cc:288] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [CANDIDATE]: Term 1 election: Requested vote from peers 638ddde6c6404ba28bbfb76554283327 (127.21.236.130:37513), fc43b02226bc45b5a2ff5e22ccf1f2fb (127.21.236.131:33701)
I20260811 02:46:45.740571 24993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aaeb76c68ef424f983998b349501447" candidate_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb"
I20260811 02:46:45.740710 24993 raft_consensus.cc:3037] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:45.740767 24753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aaeb76c68ef424f983998b349501447" candidate_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "638ddde6c6404ba28bbfb76554283327"
I20260811 02:46:45.740962 24753 raft_consensus.cc:3037] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:45.741832 24993 raft_consensus.cc:2454] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5eeccdf0c4eb49b69c037c898fab72b7 in term 1.
I20260811 02:46:45.742077 24753 raft_consensus.cc:2454] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5eeccdf0c4eb49b69c037c898fab72b7 in term 1.
I20260811 02:46:45.742242 24404 leader_election.cc:302] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5eeccdf0c4eb49b69c037c898fab72b7, fc43b02226bc45b5a2ff5e22ccf1f2fb; no voters:
I20260811 02:46:45.742456 25145 raft_consensus.cc:2781] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:45.742735 25145 raft_consensus.cc:686] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 1 LEADER]: Becoming Leader. State: Replica: 5eeccdf0c4eb49b69c037c898fab72b7, State: Running, Role: LEADER
I20260811 02:46:45.743346 25145 consensus_queue.cc:237] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } }
I20260811 02:46:45.744895 25208 tablet_replica.cc:439] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7: TxnStatusTablet state changed. Reason: New leader 5eeccdf0c4eb49b69c037c898fab72b7. Latest consensus state: current_term: 1 leader_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } }
I20260811 02:46:45.745039 25208 tablet_replica.cc:442] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:46:45.745394 25211 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 02:46:45.745605 25211 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 02:46:45.746320 24253 catalog_manager.cc:5495] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5eeccdf0c4eb49b69c037c898fab72b7 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } health_report { overall_health: UNKNOWN } } }
I20260811 02:46:45.800197 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.796321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:34717 (local address 127.0.0.1:40536)
0811 02:46:45.796487 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.796507 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.796628 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.796995 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.796998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.797006 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:46:45.797121 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.797126 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.797867 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.797871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.798477 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.798485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.798971 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.798975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.799118 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.799175 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:46:45.799876 (+ 701us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:46:45.799892 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.799963 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.800051 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 02:46:45.800197 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.796463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.21.236.129:34717)
0811 02:46:45.796594 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.796599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.796615 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:46:45.796811 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.796813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.796873 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.796979 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:46:45.797237 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.797750 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.798626 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.798859 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.798969 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.799340 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:46:45.799774 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:46:45.799905 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.799982 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.800042 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 02:46:45.803570 24993 raft_consensus.cc:1261] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 1 FOLLOWER]: Refusing update from remote peer 5eeccdf0c4eb49b69c037c898fab72b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:45.803705 24753 raft_consensus.cc:1261] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327 [term 1 FOLLOWER]: Refusing update from remote peer 5eeccdf0c4eb49b69c037c898fab72b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:45.804101 25145 consensus_queue.cc:1035] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:46:45.804303 25208 consensus_queue.cc:1035] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:46:45.805735 25144 tablet_replica.cc:439] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb: TxnStatusTablet state changed. Reason: New leader 5eeccdf0c4eb49b69c037c898fab72b7. Latest consensus state: current_term: 1 leader_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } }
I20260811 02:46:45.805851 25144 tablet_replica.cc:442] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:46:45.805964 25169 tablet_replica.cc:439] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327: TxnStatusTablet state changed. Reason: New leader 5eeccdf0c4eb49b69c037c898fab72b7. Latest consensus state: current_term: 1 leader_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } }
I20260811 02:46:45.806058 25169 tablet_replica.cc:442] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:46:45.808292 25208 tablet_replica.cc:439] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } }
I20260811 02:46:45.808405 25208 tablet_replica.cc:442] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:46:45.808900 25144 tablet_replica.cc:439] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } }
I20260811 02:46:45.808995 25144 tablet_replica.cc:442] T 6aaeb76c68ef424f983998b349501447 P fc43b02226bc45b5a2ff5e22ccf1f2fb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:46:45.809643 25145 tablet_replica.cc:439] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } }
I20260811 02:46:45.809752 25145 tablet_replica.cc:442] T 6aaeb76c68ef424f983998b349501447 P 5eeccdf0c4eb49b69c037c898fab72b7: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:46:45.811591 25169 tablet_replica.cc:439] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 } } peers { permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 } } peers { permanent_uuid: "fc43b02226bc45b5a2ff5e22ccf1f2fb" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33701 } } }
I20260811 02:46:45.811709 25169 tablet_replica.cc:442] T 6aaeb76c68ef424f983998b349501447 P 638ddde6c6404ba28bbfb76554283327: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:46:45.812373 25161 mvcc.cc:205] Tried to move back new op lower bound from 7317161598165438464 to 7317161597927636992. Current Snapshot: MvccSnapshot[applied={T|T < 7317161598165438464 or (T in {7317161598165438464})}]
I20260811 02:46:45.819479 25133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.815506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34134 (local address 127.21.236.131:33701)
0811 02:46:45.815653 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.815657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.815670 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:46:45.815838 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.815840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.815893 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.815977 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:46:45.816238 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.816756 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.817922 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.818145 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.818221 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.818511 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:46:45.819029 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:46:45.819185 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.819273 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.819328 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:46:45.831732 25225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.827346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:34717 (local address 127.21.236.131:36683)
0811 02:46:45.827675 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.827688 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.827780 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.828125 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.828131 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.828142 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:46:45.828340 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.828347 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.829505 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.829508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.830277 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.830287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.830859 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.830861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.830941 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.830987 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:46:45.831417 (+ 430us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:46:45.831426 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.831500 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.831586 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":113,"threads_started":1}
I20260811 02:46:45.831724 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.827541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:36683 (local address 127.21.236.129:34717)
0811 02:46:45.827683 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.827688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.827760 (+ 72us) server_negotiation.cc:407] Connection header received
0811 02:46:45.827878 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.827881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.827936 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.828017 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:46:45.828590 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.829383 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.830390 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.830735 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.830841 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.831160 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:46:45.831250 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:46:45.831325 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.831532 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.831588 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:46:45.840881 24753 raft_consensus.cc:1261] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327 [term 1 FOLLOWER]: Refusing update from remote peer fc43b02226bc45b5a2ff5e22ccf1f2fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:45.841681 24512 raft_consensus.cc:1261] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 1 FOLLOWER]: Refusing update from remote peer fc43b02226bc45b5a2ff5e22ccf1f2fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:45.841900 25144 consensus_queue.cc:1035] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "638ddde6c6404ba28bbfb76554283327" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 37513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:46:45.842572 25144 consensus_queue.cc:1035] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5eeccdf0c4eb49b69c037c898fab72b7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:46:45.866870 25230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.862773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:33701 (local address 127.21.236.129:57709)
0811 02:46:45.863189 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:45.863209 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:45.863313 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:45.863556 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:45.863558 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:45.863566 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:46:45.863726 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.863731 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.864593 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.864596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.865528 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:45.865541 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.866166 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.866169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:45.866239 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.866286 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:46:45.866587 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:46:45.866591 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 02:46:45.866631 (+ 40us) client_negotiation.cc:240] Negotiation successful
0811 02:46:45.866693 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":120,"threads_started":1}
I20260811 02:46:45.866861 25133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.863210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:57709 (local address 127.21.236.131:33701)
0811 02:46:45.863342 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.863349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.863365 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:46:45.863412 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.863415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.863465 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.863546 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:46:45.863837 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.864453 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.865672 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.866020 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.866143 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.866415 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:46:45.866498 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:46:45.866572 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.866654 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.866725 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 02:46:45.892107 24253 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:46:45.892305 24253 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:46:45.895629 24253 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00b264a7454542489aa2d99d53e3e060: Sending DeleteTablet for 3 replicas of tablet 8ae303b9f79647f1bbc5240b94b2b110
I20260811 02:46:45.896708 24721 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ae303b9f79647f1bbc5240b94b2b110 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:45 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:45150
I20260811 02:46:45.896919 25181 tablet_replica.cc:330] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327: stopping tablet replica
I20260811 02:46:45.897063 25181 raft_consensus.cc:2229] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:46:45.897203 25181 raft_consensus.cc:2258] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:45.897501 24483 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ae303b9f79647f1bbc5240b94b2b110 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:45 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:40532
I20260811 02:46:45.897711 25177 tablet_replica.cc:330] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7: stopping tablet replica
I20260811 02:46:45.897821 25177 raft_consensus.cc:2229] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:46:45.897938 25177 raft_consensus.cc:2258] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:45.899158 25177 ts_tablet_manager.cc:1905] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:45.899185 25181 ts_tablet_manager.cc:1905] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:45.902004 25177 ts_tablet_manager.cc:1918] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:46:45.902082 25177 log.cc:1196] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-0/wal/wals/8ae303b9f79647f1bbc5240b94b2b110
I20260811 02:46:45.902065 25181 ts_tablet_manager.cc:1918] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:46:45.902136 25181 log.cc:1196] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/8ae303b9f79647f1bbc5240b94b2b110
I20260811 02:46:45.902325 25177 ts_tablet_manager.cc:1939] T 8ae303b9f79647f1bbc5240b94b2b110 P 5eeccdf0c4eb49b69c037c898fab72b7: Deleting consensus metadata
I20260811 02:46:45.902386 25181 ts_tablet_manager.cc:1939] T 8ae303b9f79647f1bbc5240b94b2b110 P 638ddde6c6404ba28bbfb76554283327: Deleting consensus metadata
I20260811 02:46:45.903926 24222 catalog_manager.cc:4841] TS 5eeccdf0c4eb49b69c037c898fab72b7 (127.21.236.129:34717): tablet 8ae303b9f79647f1bbc5240b94b2b110 (table test-table [id=5bddf1929d444a94a02c7b9c2f9c3d56]) successfully deleted
I20260811 02:46:45.904120 24222 catalog_manager.cc:4841] TS 638ddde6c6404ba28bbfb76554283327 (127.21.236.130:37513): tablet 8ae303b9f79647f1bbc5240b94b2b110 (table test-table [id=5bddf1929d444a94a02c7b9c2f9c3d56]) successfully deleted
I20260811 02:46:45.907323 24959 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ae303b9f79647f1bbc5240b94b2b110 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:45 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:34132
I20260811 02:46:45.908107 25236 tablet_replica.cc:330] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb: stopping tablet replica
I20260811 02:46:45.908244 25236 raft_consensus.cc:2229] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:46:45.908565 25236 raft_consensus.cc:2258] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:45.909384 25236 ts_tablet_manager.cc:1905] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:45.912586 25236 ts_tablet_manager.cc:1918] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:46:45.912739 25236 log.cc:1196] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/8ae303b9f79647f1bbc5240b94b2b110
I20260811 02:46:45.913142 25236 ts_tablet_manager.cc:1939] T 8ae303b9f79647f1bbc5240b94b2b110 P fc43b02226bc45b5a2ff5e22ccf1f2fb: Deleting consensus metadata
I20260811 02:46:45.913967 24217 catalog_manager.cc:4841] TS fc43b02226bc45b5a2ff5e22ccf1f2fb (127.21.236.131:33701): tablet 8ae303b9f79647f1bbc5240b94b2b110 (table test-table [id=5bddf1929d444a94a02c7b9c2f9c3d56]) successfully deleted
Aug 11 02:46:45 dist-test-slave-n326 krb5kdc[24163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416405, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.236.129@KRBTEST.COM
I20260811 02:46:45.923923 25135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.907513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40540 (local address 127.21.236.129:34717)
0811 02:46:45.907652 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.907658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.911397 (+ 3739us) server_negotiation.cc:407] Connection header received
0811 02:46:45.911607 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.911610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.911661 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.911712 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:45.912754 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.913593 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.914889 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.915355 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.915466 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.919703 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:45.919715 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:45.919733 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:45.919766 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:45.921894 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:45.922434 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:45.922442 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:45.922444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:45.922540 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:45.922841 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:45.922844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:45.922846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:45.923209 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:45.923430 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.923604 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.923743 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 02:46:45.924400 24579 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40540
I20260811 02:46:45.936204 24887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:45.926213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37454 (local address 127.21.236.190:43741)
0811 02:46:45.926342 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:45.926346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:45.926787 (+ 441us) server_negotiation.cc:407] Connection header received
0811 02:46:45.926927 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:45.926930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:45.926984 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:45.927093 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:45.928077 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.928963 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.930128 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:45.930479 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:45.930596 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:45.932281 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:45.932285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:45.932288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:45.932319 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:45.934144 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:45.934780 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:45.934784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:45.934786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:45.934885 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:45.935282 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:45.935285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:45.935287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:45.935515 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:45.935663 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:45.935899 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 02:46:45.936031 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 02:46:45.936853 24253 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37454
I20260811 02:46:45.938366 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 24379
I20260811 02:46:45.953881 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 24609
I20260811 02:46:45.972405 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 24850
I20260811 02:46:45.991443 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 24187
2026-08-11T02:46:46Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4702 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:46:46 dist-test-slave-n326 krb5kdc[25249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:46:46 dist-test-slave-n326 krb5kdc[25249](info): set up 2 sockets
Aug 11 02:46:46 dist-test-slave-n326 krb5kdc[25249](info): commencing operation
krb5kdc: starting...
W20260811 02:46:47.396266 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.373s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:46:47 dist-test-slave-n326 krb5kdc[25249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:46:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:46:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:46:47.538983 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:33403
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:33403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:47.654317 25268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:47.654614 25268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:47.654656 25268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:47.658773 25268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:46:47.658886 25268 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 02:46:47.658913 25268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:47.658932 25268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:47.658949 25268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:46:47.658964 25268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:46:47.664546 25268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:33403
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:33403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:47.665659 25268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:47.666671 25268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:47.675801 25275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:47.675812 25278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:47.675822 25276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:47.676067 25268 server_base.cc:1034] running on GCE node
I20260811 02:46:47.676885 25268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:47.678068 25268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:47.679258 25268 hybrid_clock.cc:648] HybridClock initialized: now 1786416407679230 us; error 41 us; skew 500 ppm
Aug 11 02:46:47 dist-test-slave-n326 krb5kdc[25249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416407, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:47.682523 25268 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:46:47.684067 25268 webserver.cc:460] Webserver started at http://127.21.236.190:33205/ using document root <none> and password file <none>
I20260811 02:46:47.685149 25268 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:47.685276 25268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:47.685564 25268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:47.688038 25268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "a8d8bca6382f45dfad7fa09504f694d9"
format_stamp: "Formatted at 2026-08-11 02:46:47 on dist-test-slave-n326"
server_key: "0fab15984f0d7a9a9341266959fe129d"
server_key_iv: "62c5e46a2700463ee3867ed6ffb9c95b"
server_key_version: "encryptionkey@0"
I20260811 02:46:47.688697 25268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "a8d8bca6382f45dfad7fa09504f694d9"
format_stamp: "Formatted at 2026-08-11 02:46:47 on dist-test-slave-n326"
server_key: "0fab15984f0d7a9a9341266959fe129d"
server_key_iv: "62c5e46a2700463ee3867ed6ffb9c95b"
server_key_version: "encryptionkey@0"
I20260811 02:46:47.693007 25268 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:46:47.695873 25290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:47.697345 25268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:46:47.697518 25268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "a8d8bca6382f45dfad7fa09504f694d9"
format_stamp: "Formatted at 2026-08-11 02:46:47 on dist-test-slave-n326"
server_key: "0fab15984f0d7a9a9341266959fe129d"
server_key_iv: "62c5e46a2700463ee3867ed6ffb9c95b"
server_key_version: "encryptionkey@0"
I20260811 02:46:47.697672 25268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:47.714139 25268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:47.717905 25268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:47.718199 25268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:47.727034 25268 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:33403
I20260811 02:46:47.727145 25388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:33403 every 8 connection(s)
I20260811 02:46:47.728137 25268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:46:47.731191 25389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:47.737047 25389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9: Bootstrap starting.
I20260811 02:46:47.737229 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 25268
I20260811 02:46:47.737345 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:46:47.737643 22450 external_mini_cluster.cc:1428] Setting key 25813fb2652750b0b96b0c4373d438b7
I20260811 02:46:47.740032 25389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:47.740826 25389 log.cc:824] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:47.743451 25389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9: No bootstrap required, opened a new log
I20260811 02:46:47.748231 25389 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d8bca6382f45dfad7fa09504f694d9" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33403 } }
Aug 11 02:46:47 dist-test-slave-n326 krb5kdc[25249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:47.748562 25389 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:47.748637 25389 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8d8bca6382f45dfad7fa09504f694d9, State: Initialized, Role: FOLLOWER
I20260811 02:46:47.749349 25389 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d8bca6382f45dfad7fa09504f694d9" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33403 } }
I20260811 02:46:47.749526 25389 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:46:47.749615 25389 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:46:47.749720 25389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:47.750735 25389 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d8bca6382f45dfad7fa09504f694d9" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33403 } }
I20260811 02:46:47.751214 25389 leader_election.cc:302] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8d8bca6382f45dfad7fa09504f694d9; no voters:
I20260811 02:46:47.751641 25389 leader_election.cc:288] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:46:47.751781 25399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:47.752059 25399 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [term 1 LEADER]: Becoming Leader. State: Replica: a8d8bca6382f45dfad7fa09504f694d9, State: Running, Role: LEADER
I20260811 02:46:47.752417 25399 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d8bca6382f45dfad7fa09504f694d9" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33403 } }
I20260811 02:46:47.753182 25389 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:46:47.754545 25401 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8d8bca6382f45dfad7fa09504f694d9. Latest consensus state: current_term: 1 leader_uuid: "a8d8bca6382f45dfad7fa09504f694d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d8bca6382f45dfad7fa09504f694d9" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33403 } } }
I20260811 02:46:47.754712 25401 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [sys.catalog]: This master's current role is: LEADER
I20260811 02:46:47.755005 25400 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8d8bca6382f45dfad7fa09504f694d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8d8bca6382f45dfad7fa09504f694d9" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33403 } } }
I20260811 02:46:47.755159 25400 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9 [sys.catalog]: This master's current role is: LEADER
I20260811 02:46:47.756251 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:47.739270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37676 (local address 127.21.236.190:33403)
0811 02:46:47.739771 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:47.739788 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:47.739827 (+ 39us) server_negotiation.cc:407] Connection header received
0811 02:46:47.740623 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:47.740644 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:47.741073 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:47.741400 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:47.742301 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:47.743972 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:47.745200 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:47.746111 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:47.746314 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:47.749477 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:47.749530 (+ 53us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:47.749553 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:47.749606 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:47.752083 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:47.752675 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:47.752682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:47.752688 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:47.752839 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:47.753269 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:47.753273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:47.753274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:47.753824 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:47.754051 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:47.754536 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 02:46:47.754879 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":171,"threads_started":1}
W20260811 02:46:47.758741 25422 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:46:47.758838 25422 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:46:47.758906 25414 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:46:47.759940 25414 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:46:47.766702 25414 catalog_manager.cc:1347] Generated new cluster ID: d3c3cc4424d049f8936ba5e377b3c7bc
I20260811 02:46:47.766796 25414 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:46:47.789877 25414 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:46:47.791165 25414 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:46:47.800797 25414 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9: Generated new TSK 0
I20260811 02:46:47.801616 25414 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:46:47.842656 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:33403
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:47.960330 25430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:47.960599 25430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:47.960649 25430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:47.965029 25430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:47.965124 25430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:47.965174 25430 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 02:46:47.965258 25430 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:46:47.965317 25430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:46:47.971693 25430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.236.190:33403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:47.972882 25430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:47.974016 25430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:47.984910 25446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:47.984891 25440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:47.984894 25442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:47.985544 25430 server_base.cc:1034] running on GCE node
I20260811 02:46:47.986040 25430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:47.986691 25430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:47.987891 25430 hybrid_clock.cc:648] HybridClock initialized: now 1786416407987850 us; error 53 us; skew 500 ppm
Aug 11 02:46:47 dist-test-slave-n326 krb5kdc[25249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416407, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:47.991024 25430 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:46:47.992110 25430 webserver.cc:460] Webserver started at http://127.21.236.129:36981/ using document root <none> and password file <none>
I20260811 02:46:47.992798 25430 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:47.992908 25430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:47.993177 25430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:47.995568 25430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "74fea82c89ac488992fdec514870cde7"
format_stamp: "Formatted at 2026-08-11 02:46:47 on dist-test-slave-n326"
server_key: "67f9354893cd2fa1b57ae5571abdf5d1"
server_key_iv: "7b75f80e3242af3e8fcd012720b46f06"
server_key_version: "encryptionkey@0"
I20260811 02:46:47.996201 25430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "74fea82c89ac488992fdec514870cde7"
format_stamp: "Formatted at 2026-08-11 02:46:47 on dist-test-slave-n326"
server_key: "67f9354893cd2fa1b57ae5571abdf5d1"
server_key_iv: "7b75f80e3242af3e8fcd012720b46f06"
server_key_version: "encryptionkey@0"
I20260811 02:46:48.000680 25430 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:46:48.003739 25464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:48.005270 25430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:46:48.005460 25430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "74fea82c89ac488992fdec514870cde7"
format_stamp: "Formatted at 2026-08-11 02:46:47 on dist-test-slave-n326"
server_key: "67f9354893cd2fa1b57ae5571abdf5d1"
server_key_iv: "7b75f80e3242af3e8fcd012720b46f06"
server_key_version: "encryptionkey@0"
I20260811 02:46:48.005605 25430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:48.024478 25430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:48.028929 25430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:48.029167 25430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:48.031041 25430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:48.031172 25430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:48.031250 25430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:48.031291 25430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:46:48 dist-test-slave-n326 krb5kdc[25249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416407, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:48.043025 25430 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:40691
I20260811 02:46:48.043057 25652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:40691 every 8 connection(s)
I20260811 02:46:48.044278 25430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:46:48.047211 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.031894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:35367 (local address 127.21.236.190:33403)
0811 02:46:48.032084 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:48.032091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:48.033350 (+ 1259us) server_negotiation.cc:407] Connection header received
0811 02:46:48.034526 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:48.034530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:48.034600 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:48.034695 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:48.036573 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.037437 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.039548 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.039784 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.039870 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.043136 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:48.043142 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:48.043147 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:48.043179 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:48.044928 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.045667 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.045671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.045672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.045763 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.046088 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.046092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.046093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.046292 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:48.046388 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:48.046900 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 02:46:48.047024 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 02:46:48.048079 25474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.032718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:33403 (local address 127.21.236.129:35367)
0811 02:46:48.033179 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:48.033229 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:48.034281 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:48.034926 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:48.034938 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:48.035527 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:48.036343 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.036367 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.037595 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.037603 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.039410 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.039422 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.039968 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.039974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.040223 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.040979 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:48.040999 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:48.042874 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:48.045055 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:48.045063 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:48.045079 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:48.045559 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:48.045828 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:48.045831 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:48.045832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:48.045970 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:48.046408 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:48.046413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:48.046656 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 02:46:48.046823 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 02:46:48.047196 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":131,"threads_started":1}
I20260811 02:46:48.049536 25653 heartbeater.cc:347] Connected to a master server at 127.21.236.190:33403
I20260811 02:46:48.049708 25653 heartbeater.cc:464] Registering TS with master...
I20260811 02:46:48.050308 25653 heartbeater.cc:511] Master 127.21.236.190:33403 requested a full tablet report, sending...
I20260811 02:46:48.050523 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 25430
I20260811 02:46:48.050655 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:46:48.050968 22450 external_mini_cluster.cc:1428] Setting key 4dd31f62b9e7058b9f50cf7d3097dffb
I20260811 02:46:48.052203 25312 ts_manager.cc:194] Registered new tserver with Master: 74fea82c89ac488992fdec514870cde7 (127.21.236.129:40691)
I20260811 02:46:48.053958 25312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:35367
I20260811 02:46:48.060598 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.051935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:60189 (local address 127.21.236.190:33403)
0811 02:46:48.052039 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:48.052043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:48.052345 (+ 302us) server_negotiation.cc:407] Connection header received
0811 02:46:48.052489 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:48.052492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:48.052546 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:48.052636 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:48.053188 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.054059 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.055342 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.055715 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.055839 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.057132 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:48.057136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:48.057139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:48.057165 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:48.058750 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.059292 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.059296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.059298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.059387 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.059701 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.059707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.059708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.059897 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:48.060019 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:48.060293 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 02:46:48.060416 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 02:46:48.060608 25663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.051942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:33403 (local address 127.21.236.129:60189)
0811 02:46:48.052236 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:48.052252 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:48.052356 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:48.052675 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:48.052678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:48.052895 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:48.053042 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.053054 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.054275 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.054279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.055208 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.055241 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.055861 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.055865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.055970 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.056534 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:48.056554 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:48.056979 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:48.058875 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:48.058880 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:48.058883 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:48.059180 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:48.059501 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:48.059504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:48.059507 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:48.059602 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:48.060009 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:48.060013 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:48.060116 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 02:46:48.060235 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 02:46:48.060391 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":107,"threads_started":1}
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:46:48.082516 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:33403
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:48.197769 25666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:48.198079 25666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:48.198158 25666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:48.205137 25666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:48.205247 25666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:48.205315 25666 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 02:46:48.205348 25666 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:46:48.205422 25666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:46:48.211253 25666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.236.190:33403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:48.212476 25666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:48.213599 25666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:48.223937 25675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:48.223933 25674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:48.223933 25678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:48.224309 25666 server_base.cc:1034] running on GCE node
I20260811 02:46:48.225278 25666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:48.226199 25666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:48.227402 25666 hybrid_clock.cc:648] HybridClock initialized: now 1786416408227370 us; error 41 us; skew 500 ppm
Aug 11 02:46:48 dist-test-slave-n326 krb5kdc[25249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416408, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:48.231376 25666 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:46:48.234493 25666 webserver.cc:460] Webserver started at http://127.21.236.130:44047/ using document root <none> and password file <none>
I20260811 02:46:48.235262 25666 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:48.235361 25666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:48.235633 25666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:48.238113 25666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "6dcfaed1cc6b40af93594457f7c760b1"
format_stamp: "Formatted at 2026-08-11 02:46:48 on dist-test-slave-n326"
server_key: "ade328dea91ac2cebc77078cd5f6bee9"
server_key_iv: "7865eadd1cb566a6a5e890428ae515f9"
server_key_version: "encryptionkey@0"
I20260811 02:46:48.238631 25666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "6dcfaed1cc6b40af93594457f7c760b1"
format_stamp: "Formatted at 2026-08-11 02:46:48 on dist-test-slave-n326"
server_key: "ade328dea91ac2cebc77078cd5f6bee9"
server_key_iv: "7865eadd1cb566a6a5e890428ae515f9"
server_key_version: "encryptionkey@0"
I20260811 02:46:48.242623 25666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:46:48.245288 25688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:48.246549 25666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:46:48.246650 25666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "6dcfaed1cc6b40af93594457f7c760b1"
format_stamp: "Formatted at 2026-08-11 02:46:48 on dist-test-slave-n326"
server_key: "ade328dea91ac2cebc77078cd5f6bee9"
server_key_iv: "7865eadd1cb566a6a5e890428ae515f9"
server_key_version: "encryptionkey@0"
I20260811 02:46:48.246742 25666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:48.285545 25666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:48.289111 25666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:48.289278 25666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:48.290726 25666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:48.290781 25666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:48.290823 25666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:48.290889 25666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:46:48 dist-test-slave-n326 krb5kdc[25249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416408, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:48.302243 25666 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:44809
I20260811 02:46:48.302369 25870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:44809 every 8 connection(s)
I20260811 02:46:48.303334 25666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:46:48.306048 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.292289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:38713 (local address 127.21.236.190:33403)
0811 02:46:48.292462 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:48.292469 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:48.293297 (+ 828us) server_negotiation.cc:407] Connection header received
0811 02:46:48.294676 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:48.294683 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:48.294752 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:48.294863 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:48.296321 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.297229 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.298502 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.298792 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.298914 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.302039 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:48.302050 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:48.302053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:48.302079 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:48.303928 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.304567 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.304573 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.304577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.304690 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.305058 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.305061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.305063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.305285 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:48.305371 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:48.305817 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 02:46:48.305923 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 02:46:48.306872 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.292651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:33403 (local address 127.21.236.130:38713)
0811 02:46:48.293136 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:48.293188 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:48.294454 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:48.295167 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:48.295176 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:48.295662 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:48.296154 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.296169 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.297430 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.297435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.298365 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.298377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.298960 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.298964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.299204 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.299865 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:48.299881 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:48.301819 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:48.304077 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:48.304082 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:48.304096 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:48.304455 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:48.304806 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:48.304808 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:48.304809 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:48.304960 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:48.305378 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:48.305383 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:48.305605 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 02:46:48.305758 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 02:46:48.306023 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":148,"threads_started":1}
I20260811 02:46:48.308147 25874 heartbeater.cc:347] Connected to a master server at 127.21.236.190:33403
I20260811 02:46:48.308297 25874 heartbeater.cc:464] Registering TS with master...
I20260811 02:46:48.308933 25874 heartbeater.cc:511] Master 127.21.236.190:33403 requested a full tablet report, sending...
I20260811 02:46:48.310134 25312 ts_manager.cc:194] Registered new tserver with Master: 6dcfaed1cc6b40af93594457f7c760b1 (127.21.236.130:44809)
I20260811 02:46:48.310927 25312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:38713
I20260811 02:46:48.310971 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 25666
I20260811 02:46:48.311117 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:46:48.311462 22450 external_mini_cluster.cc:1428] Setting key 87c902f48330e8e4965d2da6ffdc94c3
I20260811 02:46:48.320024 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.310753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:33403 (local address 127.21.236.130:57599)
0811 02:46:48.311117 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:48.311132 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:48.311227 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:48.311651 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:48.311655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:48.311889 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:48.312008 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.312014 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.313032 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.313035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.313995 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.314010 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.314722 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.314729 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.314860 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.315521 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:48.315534 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:48.315908 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:48.318137 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:48.318140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:48.318141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:48.318470 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:48.318797 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:48.318800 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:48.318801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:48.318891 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:48.319521 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:48.319524 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:48.319612 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 02:46:48.319705 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 02:46:48.319832 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":113,"threads_started":1}
I20260811 02:46:48.320197 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.311274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:57599 (local address 127.21.236.190:33403)
0811 02:46:48.311432 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:48.311436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:48.311449 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:46:48.311491 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:48.311494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:48.311538 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:48.311622 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:48.312141 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.312868 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.314131 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.314571 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.314683 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.316088 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:48.316094 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:48.316098 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:48.316131 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:48.318004 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.318584 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.318590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.318594 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.318690 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.319001 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.319007 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.319012 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.319393 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:48.319519 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:48.319802 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 02:46:48.319948 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:46:48.343338 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:33403
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:48.461630 25889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:48.461896 25889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:48.461953 25889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:48.466044 25889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:48.466115 25889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:48.466156 25889 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 02:46:48.466171 25889 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:46:48.466248 25889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:46:48.471764 25889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.236.190:33403
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:48.472887 25889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:48.473935 25889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:48.483976 25895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:48.483958 25897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:48.484012 25889 server_base.cc:1034] running on GCE node
W20260811 02:46:48.483953 25894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:48.484603 25889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:48.485288 25889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:48.486509 25889 hybrid_clock.cc:648] HybridClock initialized: now 1786416408486477 us; error 46 us; skew 500 ppm
Aug 11 02:46:48 dist-test-slave-n326 krb5kdc[25249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416408, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:48.489629 25889 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:46:48.490583 25889 webserver.cc:460] Webserver started at http://127.21.236.131:37377/ using document root <none> and password file <none>
I20260811 02:46:48.491250 25889 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:48.491384 25889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:48.491636 25889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:48.493824 25889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "04a8a7a4ee3c46019005cd78da00a953"
format_stamp: "Formatted at 2026-08-11 02:46:48 on dist-test-slave-n326"
server_key: "eb1637e3b331e288aeaa569f5355b29d"
server_key_iv: "5b6311b55c14fbadfd0dac3bd32b8d5a"
server_key_version: "encryptionkey@0"
I20260811 02:46:48.494385 25889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "04a8a7a4ee3c46019005cd78da00a953"
format_stamp: "Formatted at 2026-08-11 02:46:48 on dist-test-slave-n326"
server_key: "eb1637e3b331e288aeaa569f5355b29d"
server_key_iv: "5b6311b55c14fbadfd0dac3bd32b8d5a"
server_key_version: "encryptionkey@0"
I20260811 02:46:48.498420 25889 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:46:48.501220 25907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:48.502822 25889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 02:46:48.502962 25889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "04a8a7a4ee3c46019005cd78da00a953"
format_stamp: "Formatted at 2026-08-11 02:46:48 on dist-test-slave-n326"
server_key: "eb1637e3b331e288aeaa569f5355b29d"
server_key_iv: "5b6311b55c14fbadfd0dac3bd32b8d5a"
server_key_version: "encryptionkey@0"
I20260811 02:46:48.503151 25889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:48.523492 25889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:48.526850 25889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:48.527164 25889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:48.528771 25889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:48.528857 25889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:48.528934 25889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:48.528990 25889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:46:48 dist-test-slave-n326 krb5kdc[25249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416408, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:48.541453 25889 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:33909
I20260811 02:46:48.541476 26118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:33909 every 8 connection(s)
I20260811 02:46:48.542418 25889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:46:48.545431 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.529815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:41207 (local address 127.21.236.190:33403)
0811 02:46:48.529970 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:48.529974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:48.531235 (+ 1261us) server_negotiation.cc:407] Connection header received
0811 02:46:48.532329 (+ 1094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:48.532333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:48.532418 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:48.532484 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:48.534344 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.535369 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.537047 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.537508 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.537665 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.541075 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:48.541080 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:48.541086 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:48.541127 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:48.543014 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.543683 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.543687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.543689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.543800 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.544250 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.544254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.544256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.544463 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:48.544582 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:48.545114 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 02:46:48.545248 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:46:48.546489 25920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.530406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:33403 (local address 127.21.236.131:41207)
0811 02:46:48.531017 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:48.531112 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:48.532055 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:48.532977 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:48.532986 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:48.533521 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:48.534171 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.534194 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.535531 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.535535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.536906 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.536917 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.537742 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.537748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.537998 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.538734 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:48.538752 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:48.540859 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:48.543179 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:48.543185 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:48.543200 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:48.543575 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:48.543922 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:48.543926 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:48.543930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:48.544120 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:48.544593 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:48.544603 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:48.544851 (+ 248us) client_negotiation.cc:769] Sending connection context
0811 02:46:48.545062 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 02:46:48.545399 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":139,"threads_started":1}
I20260811 02:46:48.548048 26119 heartbeater.cc:347] Connected to a master server at 127.21.236.190:33403
I20260811 02:46:48.548219 26119 heartbeater.cc:464] Registering TS with master...
I20260811 02:46:48.548833 26119 heartbeater.cc:511] Master 127.21.236.190:33403 requested a full tablet report, sending...
I20260811 02:46:48.550076 25320 ts_manager.cc:194] Registered new tserver with Master: 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909)
I20260811 02:46:48.550906 25320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:41207
I20260811 02:46:48.551499 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 25889
I20260811 02:46:48.551640 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:46:48.552016 22450 external_mini_cluster.cc:1428] Setting key c13c1dc9991bc8a284807cb5797f98b7
I20260811 02:46:48.554241 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:46:48.560191 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.550703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:54081 (local address 127.21.236.190:33403)
0811 02:46:48.550844 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:48.550848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:48.551134 (+ 286us) server_negotiation.cc:407] Connection header received
0811 02:46:48.551414 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:48.551418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:48.551493 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:48.551555 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:48.552316 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.553152 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.554371 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.554717 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.554809 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.556236 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:48.556239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:48.556241 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:48.556265 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:48.558362 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.558822 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.558828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.558830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.558935 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.559300 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.559305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.559307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.559507 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:48.559627 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:48.559912 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 02:46:48.560033 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:46:48.560205 26126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.550624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:33403 (local address 127.21.236.131:54081)
0811 02:46:48.551004 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:48.551039 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:48.551232 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:48.551727 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:48.551730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:48.552001 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:48.552163 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.552173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.553351 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.553354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.554226 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.554237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.554845 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.554849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.554968 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.555515 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:48.555536 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:48.556071 (+ 535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:48.558487 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:48.558489 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:48.558491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:48.558698 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:48.559053 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:48.559057 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:48.559059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:48.559193 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:48.559616 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:48.559619 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:48.559727 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 02:46:48.559839 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 02:46:48.560011 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":124,"threads_started":1}
Aug 11 02:46:48 dist-test-slave-n326 krb5kdc[25249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:46:48 dist-test-slave-n326 krb5kdc[25249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:48.584482 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.576149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37684 (local address 127.21.236.190:33403)
0811 02:46:48.576277 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:48.576280 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:48.576292 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:46:48.576329 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:48.576331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:48.576367 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:48.576458 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:48.577166 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.577682 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.578515 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.578737 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.578843 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.581431 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:48.581434 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:48.581437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:48.581461 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:48.582711 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.583234 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.583237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.583239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.583321 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.583605 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.583608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.583610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.583790 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:48.583935 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:48.584192 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 02:46:48.584307 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:46:48.587456 25320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:46:48.590427 25320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:46:48.605408 26144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.599013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58338 (local address 127.21.236.129:40691)
0811 02:46:48.600852 (+ 1839us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:48.600857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:48.600875 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:46:48.600935 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:48.600939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:48.601071 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:48.601292 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:48.601575 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.602553 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.604052 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.604669 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.604960 (+ 291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.605100 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:48.605153 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 02:46:48.605257 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1755,"thread_start_us":122,"threads_started":1}
I20260811 02:46:48.605425 26146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.599010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:44809 (local address 127.0.0.1:55280)
0811 02:46:48.599793 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:48.599810 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:48.599906 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:48.600565 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:48.600574 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:48.600601 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:48.600760 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.600766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.602206 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.602210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.603697 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.603708 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.604792 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.604799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.604947 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.604991 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 02:46:48.605104 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 02:46:48.605272 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":88,"thread_start_us":133,"threads_started":1}
I20260811 02:46:48.605433 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.598834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:40691 (local address 127.0.0.1:58338)
0811 02:46:48.599508 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:48.599558 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:48.599697 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:48.601268 (+ 1571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:48.601271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:48.601291 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:48.601413 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.601424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.602745 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.602748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.603923 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.603937 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.604819 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.604822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.605023 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.605048 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 02:46:48.605116 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 02:46:48.605267 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":196,"thread_start_us":95,"threads_started":1}
I20260811 02:46:48.605795 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.599267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:33909 (local address 127.0.0.1:42938)
0811 02:46:48.600535 (+ 1268us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:48.600548 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:48.600661 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:48.601459 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:48.601464 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:48.601486 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:48.601634 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.601642 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.602936 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.602940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.604130 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.604140 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.605363 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.605367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.605455 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.605482 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 02:46:48.605562 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 02:46:48.605646 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1189,"mutex_wait_us":25,"thread_start_us":93,"threads_started":1}
I20260811 02:46:48.606019 26147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.599689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55280 (local address 127.21.236.130:44809)
0811 02:46:48.600123 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:48.600132 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:48.600151 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:46:48.600215 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:48.600219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:48.600373 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:48.600559 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:48.600876 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.602081 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.603830 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.604655 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.604795 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.604928 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:48.605778 (+ 850us) server_negotiation.cc:299] Negotiation successful
0811 02:46:48.605883 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":143,"threads_started":1}
I20260811 02:46:48.606906 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.599950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42938 (local address 127.21.236.131:33909)
0811 02:46:48.600266 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:48.600278 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:48.600901 (+ 623us) server_negotiation.cc:407] Connection header received
0811 02:46:48.600986 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:48.600991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:48.601195 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:48.601338 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:48.601745 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.602800 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.604257 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.605027 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.605185 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.605301 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:48.606013 (+ 712us) server_negotiation.cc:299] Negotiation successful
0811 02:46:48.606723 (+ 710us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":134,"threads_started":1}
I20260811 02:46:48.608834 25772 tablet_service.cc:1467] Processing CreateTablet for tablet 05a5efbbb1ae4e5689bc013264921a9c (DEFAULT_TABLE table=test-table [id=11129f26abeb48ebb4648f046b0f42c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:48.608871 25542 tablet_service.cc:1467] Processing CreateTablet for tablet 05a5efbbb1ae4e5689bc013264921a9c (DEFAULT_TABLE table=test-table [id=11129f26abeb48ebb4648f046b0f42c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:48.609920 25772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a5efbbb1ae4e5689bc013264921a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:48.609539 26005 tablet_service.cc:1467] Processing CreateTablet for tablet 05a5efbbb1ae4e5689bc013264921a9c (DEFAULT_TABLE table=test-table [id=11129f26abeb48ebb4648f046b0f42c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:48.609977 25542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a5efbbb1ae4e5689bc013264921a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:48.610510 26005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05a5efbbb1ae4e5689bc013264921a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:48.616871 26154 tablet_bootstrap.cc:492] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1: Bootstrap starting.
I20260811 02:46:48.618399 26155 tablet_bootstrap.cc:492] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7: Bootstrap starting.
I20260811 02:46:48.619241 26154 tablet_bootstrap.cc:654] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:48.619469 26156 tablet_bootstrap.cc:492] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953: Bootstrap starting.
I20260811 02:46:48.620075 26154 log.cc:824] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:48.620692 26155 tablet_bootstrap.cc:654] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:48.621637 26155 log.cc:824] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:48.621707 26156 tablet_bootstrap.cc:654] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:48.622318 26154 tablet_bootstrap.cc:492] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1: No bootstrap required, opened a new log
I20260811 02:46:48.622535 26154 ts_tablet_manager.cc:1397] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 02:46:48.622669 26156 log.cc:824] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:48.623749 26155 tablet_bootstrap.cc:492] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7: No bootstrap required, opened a new log
I20260811 02:46:48.623939 26155 ts_tablet_manager.cc:1397] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260811 02:46:48.624691 26156 tablet_bootstrap.cc:492] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953: No bootstrap required, opened a new log
I20260811 02:46:48.624907 26156 ts_tablet_manager.cc:1397] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:46:48.626730 26154 raft_consensus.cc:348] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.627188 26154 raft_consensus.cc:374] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:48.627261 26154 raft_consensus.cc:729] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dcfaed1cc6b40af93594457f7c760b1, State: Initialized, Role: FOLLOWER
I20260811 02:46:48.627825 26154 consensus_queue.cc:260] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.628523 26156 raft_consensus.cc:348] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.628832 26156 raft_consensus.cc:374] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:48.628720 26155 raft_consensus.cc:348] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.628901 26156 raft_consensus.cc:729] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a8a7a4ee3c46019005cd78da00a953, State: Initialized, Role: FOLLOWER
I20260811 02:46:48.628974 26155 raft_consensus.cc:374] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:48.629041 26154 ts_tablet_manager.cc:1428] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:46:48.629033 26155 raft_consensus.cc:729] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74fea82c89ac488992fdec514870cde7, State: Initialized, Role: FOLLOWER
I20260811 02:46:48.629415 25874 heartbeater.cc:503] Master 127.21.236.190:33403 was elected leader, sending a full tablet report...
I20260811 02:46:48.629452 26155 consensus_queue.cc:260] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.629423 26156 consensus_queue.cc:260] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.630398 26155 ts_tablet_manager.cc:1428] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260811 02:46:48.630460 26156 ts_tablet_manager.cc:1428] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 02:46:48.630499 25653 heartbeater.cc:503] Master 127.21.236.190:33403 was elected leader, sending a full tablet report...
I20260811 02:46:48.630858 26119 heartbeater.cc:503] Master 127.21.236.190:33403 was elected leader, sending a full tablet report...
I20260811 02:46:48.676080 26165 raft_consensus.cc:482] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:46:48.676378 26165 raft_consensus.cc:504] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.677598 26165 leader_election.cc:288] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dcfaed1cc6b40af93594457f7c760b1 (127.21.236.130:44809), 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909)
I20260811 02:46:48.682554 26147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.678182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:33873 (local address 127.21.236.130:44809)
0811 02:46:48.678313 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:48.678317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:48.678604 (+ 287us) server_negotiation.cc:407] Connection header received
0811 02:46:48.678831 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:48.678833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:48.678920 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:48.679002 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:48.679346 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.680017 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.681498 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.681916 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.682058 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.682129 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:48.682331 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 02:46:48.682416 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:46:48.682587 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.678069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:58691 (local address 127.21.236.131:33909)
0811 02:46:48.678237 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:48.678242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:48.678546 (+ 304us) server_negotiation.cc:407] Connection header received
0811 02:46:48.678732 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:48.678738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:48.678797 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:48.678868 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:48.679235 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.679830 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.681479 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.681900 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.682076 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.682131 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:48.682395 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 02:46:48.682476 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 02:46:48.682582 26174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.677955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:33909 (local address 127.21.236.129:58691)
0811 02:46:48.678431 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:48.678445 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:48.678586 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:48.678912 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:48.678916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:48.678933 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:48.679104 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.679114 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.679951 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.679956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.681302 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.681314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.682083 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.682087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.682235 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.682254 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 02:46:48.682354 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 02:46:48.682423 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":38,"thread_start_us":143,"threads_started":1}
I20260811 02:46:48.682595 26168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.677955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:44809 (local address 127.21.236.129:33873)
0811 02:46:48.678526 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:48.678538 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:48.678685 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:48.679031 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:48.679033 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:48.679048 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:48.679227 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.679234 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.680135 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.680138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.681302 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.681314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.682058 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.682062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.682169 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.682223 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 02:46:48.682289 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 02:46:48.682388 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":74,"thread_start_us":148,"threads_started":1}
I20260811 02:46:48.683442 26038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a5efbbb1ae4e5689bc013264921a9c" candidate_uuid: "74fea82c89ac488992fdec514870cde7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a8a7a4ee3c46019005cd78da00a953" is_pre_election: true
I20260811 02:46:48.683516 25803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a5efbbb1ae4e5689bc013264921a9c" candidate_uuid: "74fea82c89ac488992fdec514870cde7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dcfaed1cc6b40af93594457f7c760b1" is_pre_election: true
I20260811 02:46:48.683784 26038 raft_consensus.cc:2454] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74fea82c89ac488992fdec514870cde7 in term 0.
I20260811 02:46:48.683884 25803 raft_consensus.cc:2454] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74fea82c89ac488992fdec514870cde7 in term 0.
I20260811 02:46:48.684337 25468 leader_election.cc:302] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a8a7a4ee3c46019005cd78da00a953, 74fea82c89ac488992fdec514870cde7; no voters:
I20260811 02:46:48.684633 26165 raft_consensus.cc:2781] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:46:48.684727 26165 raft_consensus.cc:482] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:46:48.684772 26165 raft_consensus.cc:3037] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:48.685928 26165 raft_consensus.cc:504] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.686415 26165 leader_election.cc:288] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [CANDIDATE]: Term 1 election: Requested vote from peers 6dcfaed1cc6b40af93594457f7c760b1 (127.21.236.130:44809), 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909)
I20260811 02:46:48.686836 26038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a5efbbb1ae4e5689bc013264921a9c" candidate_uuid: "74fea82c89ac488992fdec514870cde7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a8a7a4ee3c46019005cd78da00a953"
I20260811 02:46:48.686870 25803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05a5efbbb1ae4e5689bc013264921a9c" candidate_uuid: "74fea82c89ac488992fdec514870cde7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dcfaed1cc6b40af93594457f7c760b1"
I20260811 02:46:48.686995 26038 raft_consensus.cc:3037] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:48.687005 25803 raft_consensus.cc:3037] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:48.688587 25803 raft_consensus.cc:2454] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74fea82c89ac488992fdec514870cde7 in term 1.
I20260811 02:46:48.688861 26038 raft_consensus.cc:2454] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74fea82c89ac488992fdec514870cde7 in term 1.
I20260811 02:46:48.688992 25467 leader_election.cc:302] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dcfaed1cc6b40af93594457f7c760b1, 74fea82c89ac488992fdec514870cde7; no voters:
I20260811 02:46:48.689225 26165 raft_consensus.cc:2781] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:48.689503 26165 raft_consensus.cc:686] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [term 1 LEADER]: Becoming Leader. State: Replica: 74fea82c89ac488992fdec514870cde7, State: Running, Role: LEADER
I20260811 02:46:48.689906 26165 consensus_queue.cc:237] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.693524 25312 catalog_manager.cc:5495] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74fea82c89ac488992fdec514870cde7 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "74fea82c89ac488992fdec514870cde7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } health_report { overall_health: UNKNOWN } } }
Aug 11 02:46:48 dist-test-slave-n326 krb5kdc[25249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416407, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:48.755738 26185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.745037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:33403 (local address 127.0.0.1:37688)
0811 02:46:48.745425 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:48.745441 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:48.745565 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:48.745917 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:48.745919 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:48.746524 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:48.746642 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.746648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.747914 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.747917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.748612 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:48.748626 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.749280 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.749283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:48.749373 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.750115 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:48.750127 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:48.751831 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:48.753810 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:48.753820 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:48.753827 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:48.754167 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:48.754596 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:48.754599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:48.754600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:48.754691 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:48.755219 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:48.755225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:48.755341 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 02:46:48.755449 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 02:46:48.755591 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":142,"threads_started":1}
I20260811 02:46:48.755780 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:48.745097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37688 (local address 127.21.236.190:33403)
0811 02:46:48.745254 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:48.745260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:48.745534 (+ 274us) server_negotiation.cc:407] Connection header received
0811 02:46:48.745698 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:48.745701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:48.745750 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:48.745836 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:48.746952 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.747776 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.748730 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:48.749126 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:48.749238 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:48.752008 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:48.752012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:48.752014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:48.752036 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:48.753687 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.754310 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.754317 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.754319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.754425 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:48.754816 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:48.754819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:48.754820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:48.755038 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:48.755200 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:48.755562 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 02:46:48.755670 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 02:46:48.760357 25312 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:37688:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 02:46:48.761272 25312 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:46:48.766538 25772 tablet_service.cc:1467] Processing CreateTablet for tablet bc553b1b7a6e43e68f9ec1058e2be847 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb8547613d0c42b5b303615c3db6276e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:46:48.766592 26005 tablet_service.cc:1467] Processing CreateTablet for tablet bc553b1b7a6e43e68f9ec1058e2be847 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb8547613d0c42b5b303615c3db6276e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:46:48.766580 25542 tablet_service.cc:1467] Processing CreateTablet for tablet bc553b1b7a6e43e68f9ec1058e2be847 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb8547613d0c42b5b303615c3db6276e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:46:48.767295 25772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc553b1b7a6e43e68f9ec1058e2be847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:48.767297 26005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc553b1b7a6e43e68f9ec1058e2be847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:48.767297 25542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc553b1b7a6e43e68f9ec1058e2be847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:48.770118 26154 tablet_bootstrap.cc:492] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1: Bootstrap starting.
I20260811 02:46:48.770232 26156 tablet_bootstrap.cc:492] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953: Bootstrap starting.
I20260811 02:46:48.770581 26155 tablet_bootstrap.cc:492] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7: Bootstrap starting.
I20260811 02:46:48.771310 26154 tablet_bootstrap.cc:654] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:48.771340 26156 tablet_bootstrap.cc:654] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:48.772068 26155 tablet_bootstrap.cc:654] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:48.772652 26154 tablet_bootstrap.cc:492] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1: No bootstrap required, opened a new log
I20260811 02:46:48.772769 26154 ts_tablet_manager.cc:1397] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:46:48.773204 26155 tablet_bootstrap.cc:492] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7: No bootstrap required, opened a new log
I20260811 02:46:48.773208 26156 tablet_bootstrap.cc:492] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953: No bootstrap required, opened a new log
I20260811 02:46:48.773262 26155 ts_tablet_manager.cc:1397] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 02:46:48.773288 26156 ts_tablet_manager.cc:1397] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:46:48.773262 26154 raft_consensus.cc:348] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.773380 26154 raft_consensus.cc:374] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:48.773419 26154 raft_consensus.cc:729] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dcfaed1cc6b40af93594457f7c760b1, State: Initialized, Role: FOLLOWER
I20260811 02:46:48.773541 26154 consensus_queue.cc:260] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.773726 26156 raft_consensus.cc:348] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.773732 26155 raft_consensus.cc:348] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.773823 26156 raft_consensus.cc:374] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:48.773847 26154 ts_tablet_manager.cc:1428] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:46:48.773857 26155 raft_consensus.cc:374] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:48.773859 26156 raft_consensus.cc:729] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a8a7a4ee3c46019005cd78da00a953, State: Initialized, Role: FOLLOWER
I20260811 02:46:48.773888 26155 raft_consensus.cc:729] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74fea82c89ac488992fdec514870cde7, State: Initialized, Role: FOLLOWER
I20260811 02:46:48.773986 26156 consensus_queue.cc:260] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.774041 26155 consensus_queue.cc:260] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:48.774122 26162 tablet_replica.cc:439] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } } }
I20260811 02:46:48.774276 26162 tablet_replica.cc:442] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:46:48.774308 26165 tablet_replica.cc:439] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } } }
I20260811 02:46:48.774324 26166 tablet_replica.cc:439] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } } }
I20260811 02:46:48.774446 26165 tablet_replica.cc:442] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:46:48.774469 26166 tablet_replica.cc:442] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:46:48.774340 26156 ts_tablet_manager.cc:1428] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:46:48.775410 26155 ts_tablet_manager.cc:1428] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
W20260811 02:46:48.793594 26120 tablet.cc:2369] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:46:48.796057 25654 tablet.cc:2369] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:46:48.804805 25875 tablet.cc:2369] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:46:49.003657 26165 raft_consensus.cc:482] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:46:49.003834 26165 raft_consensus.cc:504] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:49.004315 26165 leader_election.cc:288] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dcfaed1cc6b40af93594457f7c760b1 (127.21.236.130:44809), 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909)
I20260811 02:46:49.004819 26038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc553b1b7a6e43e68f9ec1058e2be847" candidate_uuid: "74fea82c89ac488992fdec514870cde7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a8a7a4ee3c46019005cd78da00a953" is_pre_election: true
I20260811 02:46:49.004819 25803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc553b1b7a6e43e68f9ec1058e2be847" candidate_uuid: "74fea82c89ac488992fdec514870cde7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dcfaed1cc6b40af93594457f7c760b1" is_pre_election: true
I20260811 02:46:49.004989 26038 raft_consensus.cc:2454] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74fea82c89ac488992fdec514870cde7 in term 0.
I20260811 02:46:49.005000 25803 raft_consensus.cc:2454] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74fea82c89ac488992fdec514870cde7 in term 0.
I20260811 02:46:49.005321 25468 leader_election.cc:302] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a8a7a4ee3c46019005cd78da00a953, 74fea82c89ac488992fdec514870cde7; no voters:
I20260811 02:46:49.005518 26165 raft_consensus.cc:2781] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:46:49.005625 26165 raft_consensus.cc:482] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:46:49.005681 26165 raft_consensus.cc:3037] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:49.007308 26165 raft_consensus.cc:504] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:49.007735 26165 leader_election.cc:288] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [CANDIDATE]: Term 1 election: Requested vote from peers 6dcfaed1cc6b40af93594457f7c760b1 (127.21.236.130:44809), 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909)
I20260811 02:46:49.008190 26038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc553b1b7a6e43e68f9ec1058e2be847" candidate_uuid: "74fea82c89ac488992fdec514870cde7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a8a7a4ee3c46019005cd78da00a953"
I20260811 02:46:49.008225 25803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc553b1b7a6e43e68f9ec1058e2be847" candidate_uuid: "74fea82c89ac488992fdec514870cde7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dcfaed1cc6b40af93594457f7c760b1"
I20260811 02:46:49.008350 25803 raft_consensus.cc:3037] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:49.008350 26038 raft_consensus.cc:3037] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:49.009539 25803 raft_consensus.cc:2454] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74fea82c89ac488992fdec514870cde7 in term 1.
I20260811 02:46:49.009539 26038 raft_consensus.cc:2454] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74fea82c89ac488992fdec514870cde7 in term 1.
I20260811 02:46:49.009894 25467 leader_election.cc:302] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dcfaed1cc6b40af93594457f7c760b1, 74fea82c89ac488992fdec514870cde7; no voters:
I20260811 02:46:49.010128 26165 raft_consensus.cc:2781] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:49.010262 26165 raft_consensus.cc:686] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [term 1 LEADER]: Becoming Leader. State: Replica: 74fea82c89ac488992fdec514870cde7, State: Running, Role: LEADER
I20260811 02:46:49.010416 26165 consensus_queue.cc:237] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:49.011189 26176 tablet_replica.cc:439] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7: TxnStatusTablet state changed. Reason: New leader 74fea82c89ac488992fdec514870cde7. Latest consensus state: current_term: 1 leader_uuid: "74fea82c89ac488992fdec514870cde7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } } }
I20260811 02:46:49.011312 26176 tablet_replica.cc:442] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:46:49.011665 26192 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 02:46:49.011910 26192 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 02:46:49.012382 25316 catalog_manager.cc:5495] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74fea82c89ac488992fdec514870cde7 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "74fea82c89ac488992fdec514870cde7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260811 02:46:49.075330 26185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:49.070610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:40691 (local address 127.0.0.1:58354)
0811 02:46:49.070811 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:49.070830 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:49.070974 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:49.071333 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:49.071335 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:49.071343 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:46:49.071451 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:49.071458 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:49.072513 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:49.072516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:49.073255 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:49.073264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:49.073899 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:49.073903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:49.074022 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:49.074086 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:46:49.074951 (+ 865us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:46:49.074967 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 02:46:49.075034 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:46:49.075173 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 02:46:49.075330 26144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:49.070815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58354 (local address 127.21.236.129:40691)
0811 02:46:49.070979 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:49.070985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:49.071002 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:46:49.071123 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:49.071126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:49.071194 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:49.071298 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:46:49.071563 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:49.072354 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:49.073380 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:49.073755 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:49.073872 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:49.074279 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:46:49.074824 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:46:49.074956 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:49.075049 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 02:46:49.075134 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 02:46:49.081209 26038 raft_consensus.cc:1261] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Refusing update from remote peer 74fea82c89ac488992fdec514870cde7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:49.081256 25803 raft_consensus.cc:1261] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Refusing update from remote peer 74fea82c89ac488992fdec514870cde7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:49.082032 26176 consensus_queue.cc:1035] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:46:49.082230 26165 consensus_queue.cc:1035] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:46:49.085809 26166 tablet_replica.cc:439] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953: TxnStatusTablet state changed. Reason: New leader 74fea82c89ac488992fdec514870cde7. Latest consensus state: current_term: 1 leader_uuid: "74fea82c89ac488992fdec514870cde7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } } }
I20260811 02:46:49.085944 26166 tablet_replica.cc:442] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:46:49.086300 26162 tablet_replica.cc:439] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1: TxnStatusTablet state changed. Reason: New leader 74fea82c89ac488992fdec514870cde7. Latest consensus state: current_term: 1 leader_uuid: "74fea82c89ac488992fdec514870cde7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } } }
I20260811 02:46:49.086421 26162 tablet_replica.cc:442] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:46:49.089982 26176 tablet_replica.cc:439] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "74fea82c89ac488992fdec514870cde7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } } }
I20260811 02:46:49.090107 26176 tablet_replica.cc:442] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:46:49.090229 26176 tablet_replica.cc:439] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "74fea82c89ac488992fdec514870cde7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } } }
I20260811 02:46:49.090319 26176 tablet_replica.cc:442] T bc553b1b7a6e43e68f9ec1058e2be847 P 74fea82c89ac488992fdec514870cde7: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:46:49.090273 26162 tablet_replica.cc:439] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "74fea82c89ac488992fdec514870cde7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } } }
I20260811 02:46:49.090384 26162 tablet_replica.cc:442] T bc553b1b7a6e43e68f9ec1058e2be847 P 6dcfaed1cc6b40af93594457f7c760b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:46:49.090767 26195 mvcc.cc:205] Tried to move back new op lower bound from 7317161611589455872 to 7317161611308249088. Current Snapshot: MvccSnapshot[applied={T|T < 7317161611589455872}]
I20260811 02:46:49.091095 26197 mvcc.cc:205] Tried to move back new op lower bound from 7317161611589455872 to 7317161611308249088. Current Snapshot: MvccSnapshot[applied={T|T < 7317161611589455872}]
I20260811 02:46:49.092340 26166 tablet_replica.cc:439] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "74fea82c89ac488992fdec514870cde7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } } }
I20260811 02:46:49.092496 26166 tablet_replica.cc:442] T bc553b1b7a6e43e68f9ec1058e2be847 P 04a8a7a4ee3c46019005cd78da00a953: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:46:49.092782 26196 mvcc.cc:205] Tried to move back new op lower bound from 7317161611589455872 to 7317161611308249088. Current Snapshot: MvccSnapshot[applied={T|T < 7317161611589455872}]
I20260811 02:46:49.102537 26144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:49.098418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58360 (local address 127.21.236.129:40691)
0811 02:46:49.098583 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:49.098588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:49.098813 (+ 225us) server_negotiation.cc:407] Connection header received
0811 02:46:49.098968 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:49.098971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:49.099017 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:49.099142 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:46:49.099407 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:49.099991 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:49.101222 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:49.101573 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:49.101671 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:49.102004 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:46:49.102082 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:46:49.102157 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:49.102267 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 02:46:49.102315 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 02:46:49.114629 26144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:49.111096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:55471 (local address 127.21.236.129:40691)
0811 02:46:49.111231 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:49.111236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:49.111439 (+ 203us) server_negotiation.cc:407] Connection header received
0811 02:46:49.111547 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:49.111550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:49.111593 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:49.111660 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:46:49.111917 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:49.112441 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:49.113593 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:49.113803 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:49.113891 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:49.114215 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:46:49.114280 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:46:49.114338 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:49.114451 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 02:46:49.114497 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":35}
I20260811 02:46:49.114683 26223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:49.111026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:40691 (local address 127.21.236.129:55471)
0811 02:46:49.111344 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:49.111359 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:49.111467 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:49.111688 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:49.111691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:49.111699 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:46:49.111822 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:49.111828 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:49.112558 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:49.112561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:49.113445 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:49.113457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:49.113923 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:49.113926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:49.114007 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:49.114053 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:46:49.114363 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:46:49.114370 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 02:46:49.114441 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 02:46:49.114526 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":18,"thread_start_us":128,"threads_started":1}
I20260811 02:46:49.121315 26038 raft_consensus.cc:1261] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Refusing update from remote peer 74fea82c89ac488992fdec514870cde7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:49.121395 25803 raft_consensus.cc:1261] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Refusing update from remote peer 74fea82c89ac488992fdec514870cde7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:49.121748 26205 consensus_queue.cc:1035] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:46:49.121886 26165 consensus_queue.cc:1035] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:46:49.167726 25320 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:46:49.167922 25320 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:46:49.170693 25320 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9: Sending DeleteTablet for 3 replicas of tablet 05a5efbbb1ae4e5689bc013264921a9c
I20260811 02:46:49.171556 25541 tablet_service.cc:1514] Processing DeleteTablet for tablet 05a5efbbb1ae4e5689bc013264921a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:49 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:58338
I20260811 02:46:49.171545 25772 tablet_service.cc:1514] Processing DeleteTablet for tablet 05a5efbbb1ae4e5689bc013264921a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:49 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:55280
I20260811 02:46:49.171587 26005 tablet_service.cc:1514] Processing DeleteTablet for tablet 05a5efbbb1ae4e5689bc013264921a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:49 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:42938
I20260811 02:46:49.172207 26230 tablet_replica.cc:330] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953: stopping tablet replica
I20260811 02:46:49.172250 26235 tablet_replica.cc:330] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1: stopping tablet replica
I20260811 02:46:49.172598 26235 raft_consensus.cc:2229] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:46:49.172588 26230 raft_consensus.cc:2229] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:46:49.172860 26235 raft_consensus.cc:2258] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:49.172868 26230 raft_consensus.cc:2258] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:49.173036 25316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:46:49.173481 25316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:46:49.174579 26235 ts_tablet_manager.cc:1905] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:49.174593 26230 ts_tablet_manager.cc:1905] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:49.174867 26233 tablet_replica.cc:330] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7: stopping tablet replica
I20260811 02:46:49.175261 26233 raft_consensus.cc:2229] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:46:49.178004 26230 ts_tablet_manager.cc:1918] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:46:49.178129 26230 log.cc:1196] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/05a5efbbb1ae4e5689bc013264921a9c
I20260811 02:46:49.178471 26230 ts_tablet_manager.cc:1939] T 05a5efbbb1ae4e5689bc013264921a9c P 04a8a7a4ee3c46019005cd78da00a953: Deleting consensus metadata
I20260811 02:46:49.178706 26233 raft_consensus.cc:2258] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:49.179044 25772 tablet_service.cc:1467] Processing CreateTablet for tablet 6db2567568d44aa8a6dd475296e79b95 (DEFAULT_TABLE table=test-table [id=2992d193f5af4d12b7c4a23773059745]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:49.179383 25772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6db2567568d44aa8a6dd475296e79b95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:49.180348 26233 ts_tablet_manager.cc:1905] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:49.180346 26235 ts_tablet_manager.cc:1918] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:46:49.180460 26235 log.cc:1196] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/05a5efbbb1ae4e5689bc013264921a9c
I20260811 02:46:49.180789 26235 ts_tablet_manager.cc:1939] T 05a5efbbb1ae4e5689bc013264921a9c P 6dcfaed1cc6b40af93594457f7c760b1: Deleting consensus metadata
I20260811 02:46:49.181650 26154 tablet_bootstrap.cc:492] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1: Bootstrap starting.
I20260811 02:46:49.181859 25293 catalog_manager.cc:4841] TS 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909): tablet 05a5efbbb1ae4e5689bc013264921a9c (table test-table [id=11129f26abeb48ebb4648f046b0f42c0]) successfully deleted
I20260811 02:46:49.182358 25292 catalog_manager.cc:4841] TS 6dcfaed1cc6b40af93594457f7c760b1 (127.21.236.130:44809): tablet 05a5efbbb1ae4e5689bc013264921a9c (table test-table [id=11129f26abeb48ebb4648f046b0f42c0]) successfully deleted
I20260811 02:46:49.182687 26154 tablet_bootstrap.cc:654] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:49.182946 26005 tablet_service.cc:1467] Processing CreateTablet for tablet 6db2567568d44aa8a6dd475296e79b95 (DEFAULT_TABLE table=test-table [id=2992d193f5af4d12b7c4a23773059745]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:49.183457 26005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6db2567568d44aa8a6dd475296e79b95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:49.183459 25541 tablet_service.cc:1467] Processing CreateTablet for tablet 6db2567568d44aa8a6dd475296e79b95 (DEFAULT_TABLE table=test-table [id=2992d193f5af4d12b7c4a23773059745]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:49.183701 25541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6db2567568d44aa8a6dd475296e79b95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:49.183952 26233 ts_tablet_manager.cc:1918] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:46:49.184041 26233 log.cc:1196] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/wal/wals/05a5efbbb1ae4e5689bc013264921a9c
I20260811 02:46:49.184330 26233 ts_tablet_manager.cc:1939] T 05a5efbbb1ae4e5689bc013264921a9c P 74fea82c89ac488992fdec514870cde7: Deleting consensus metadata
I20260811 02:46:49.185493 25291 catalog_manager.cc:4841] TS 74fea82c89ac488992fdec514870cde7 (127.21.236.129:40691): tablet 05a5efbbb1ae4e5689bc013264921a9c (table test-table [id=11129f26abeb48ebb4648f046b0f42c0]) successfully deleted
I20260811 02:46:49.185763 26155 tablet_bootstrap.cc:492] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7: Bootstrap starting.
I20260811 02:46:49.185966 26156 tablet_bootstrap.cc:492] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953: Bootstrap starting.
I20260811 02:46:49.186559 26155 tablet_bootstrap.cc:654] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:49.186859 26154 tablet_bootstrap.cc:492] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1: No bootstrap required, opened a new log
I20260811 02:46:49.186951 26156 tablet_bootstrap.cc:654] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:49.186957 26154 ts_tablet_manager.cc:1397] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 02:46:49.187453 26154 raft_consensus.cc:348] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:49.187604 26154 raft_consensus.cc:374] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:49.187649 26154 raft_consensus.cc:729] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dcfaed1cc6b40af93594457f7c760b1, State: Initialized, Role: FOLLOWER
I20260811 02:46:49.187798 26154 consensus_queue.cc:260] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:49.188076 26154 ts_tablet_manager.cc:1428] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 02:46:49.188870 26156 tablet_bootstrap.cc:492] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953: No bootstrap required, opened a new log
I20260811 02:46:49.188966 26156 ts_tablet_manager.cc:1397] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:46:49.189170 26155 tablet_bootstrap.cc:492] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7: No bootstrap required, opened a new log
I20260811 02:46:49.189268 26155 ts_tablet_manager.cc:1397] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260811 02:46:49.189437 26156 raft_consensus.cc:348] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:49.189551 26156 raft_consensus.cc:374] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:49.189589 26156 raft_consensus.cc:729] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a8a7a4ee3c46019005cd78da00a953, State: Initialized, Role: FOLLOWER
I20260811 02:46:49.189734 26156 consensus_queue.cc:260] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:49.189715 26155 raft_consensus.cc:348] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:49.189829 26155 raft_consensus.cc:374] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:49.189869 26155 raft_consensus.cc:729] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74fea82c89ac488992fdec514870cde7, State: Initialized, Role: FOLLOWER
I20260811 02:46:49.190001 26156 ts_tablet_manager.cc:1428] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:46:49.189988 26155 consensus_queue.cc:260] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:49.190268 26155 ts_tablet_manager.cc:1428] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 02:46:49.561228 26165 raft_consensus.cc:482] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:46:49.561383 26165 raft_consensus.cc:504] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:49.561920 26165 leader_election.cc:288] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6dcfaed1cc6b40af93594457f7c760b1 (127.21.236.130:44809), 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909)
I20260811 02:46:49.562464 25803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db2567568d44aa8a6dd475296e79b95" candidate_uuid: "74fea82c89ac488992fdec514870cde7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dcfaed1cc6b40af93594457f7c760b1" is_pre_election: true
I20260811 02:46:49.562638 25803 raft_consensus.cc:2454] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74fea82c89ac488992fdec514870cde7 in term 0.
I20260811 02:46:49.562631 26038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db2567568d44aa8a6dd475296e79b95" candidate_uuid: "74fea82c89ac488992fdec514870cde7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a8a7a4ee3c46019005cd78da00a953" is_pre_election: true
I20260811 02:46:49.562801 26038 raft_consensus.cc:2454] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 74fea82c89ac488992fdec514870cde7 in term 0.
I20260811 02:46:49.562963 25467 leader_election.cc:302] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dcfaed1cc6b40af93594457f7c760b1, 74fea82c89ac488992fdec514870cde7; no voters:
I20260811 02:46:49.563218 26165 raft_consensus.cc:2781] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:46:49.563289 26165 raft_consensus.cc:482] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:46:49.563318 26165 raft_consensus.cc:3037] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:49.564280 26165 raft_consensus.cc:504] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:49.564591 26165 leader_election.cc:288] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [CANDIDATE]: Term 1 election: Requested vote from peers 6dcfaed1cc6b40af93594457f7c760b1 (127.21.236.130:44809), 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909)
I20260811 02:46:49.565021 26038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db2567568d44aa8a6dd475296e79b95" candidate_uuid: "74fea82c89ac488992fdec514870cde7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a8a7a4ee3c46019005cd78da00a953"
I20260811 02:46:49.565021 25803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6db2567568d44aa8a6dd475296e79b95" candidate_uuid: "74fea82c89ac488992fdec514870cde7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6dcfaed1cc6b40af93594457f7c760b1"
I20260811 02:46:49.565172 25803 raft_consensus.cc:3037] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:49.565172 26038 raft_consensus.cc:3037] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:49.566406 26038 raft_consensus.cc:2454] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74fea82c89ac488992fdec514870cde7 in term 1.
I20260811 02:46:49.566397 25803 raft_consensus.cc:2454] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 74fea82c89ac488992fdec514870cde7 in term 1.
I20260811 02:46:49.566726 25468 leader_election.cc:302] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a8a7a4ee3c46019005cd78da00a953, 74fea82c89ac488992fdec514870cde7; no voters:
I20260811 02:46:49.566967 26165 raft_consensus.cc:2781] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:49.567122 26165 raft_consensus.cc:686] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [term 1 LEADER]: Becoming Leader. State: Replica: 74fea82c89ac488992fdec514870cde7, State: Running, Role: LEADER
I20260811 02:46:49.567279 26165 consensus_queue.cc:237] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:49.569193 25316 catalog_manager.cc:5495] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 74fea82c89ac488992fdec514870cde7 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "74fea82c89ac488992fdec514870cde7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260811 02:46:49.662053 26038 raft_consensus.cc:1261] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Refusing update from remote peer 74fea82c89ac488992fdec514870cde7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:49.662317 25803 raft_consensus.cc:1261] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Refusing update from remote peer 74fea82c89ac488992fdec514870cde7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:49.662565 26205 consensus_queue.cc:1035] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:46:49.662864 26176 consensus_queue.cc:1035] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 02:46:49.677824 26195 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 02:46:51.233326 25320 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:46:51.233522 25320 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:46:51.236261 25320 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9: Sending DeleteTablet for 3 replicas of tablet 6db2567568d44aa8a6dd475296e79b95
I20260811 02:46:51.236914 25772 tablet_service.cc:1514] Processing DeleteTablet for tablet 6db2567568d44aa8a6dd475296e79b95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:51 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:55280
I20260811 02:46:51.237004 26005 tablet_service.cc:1514] Processing DeleteTablet for tablet 6db2567568d44aa8a6dd475296e79b95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:51 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:42938
I20260811 02:46:51.236987 25542 tablet_service.cc:1514] Processing DeleteTablet for tablet 6db2567568d44aa8a6dd475296e79b95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:51 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:58338
I20260811 02:46:51.237370 26285 tablet_replica.cc:330] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7: stopping tablet replica
I20260811 02:46:51.237423 26282 tablet_replica.cc:330] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1: stopping tablet replica
I20260811 02:46:51.237563 26285 raft_consensus.cc:2229] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:46:51.237741 26282 raft_consensus.cc:2229] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:46:51.237771 26285 raft_consensus.cc:2258] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:51.237914 26282 raft_consensus.cc:2258] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:51.238035 26284 tablet_replica.cc:330] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953: stopping tablet replica
I20260811 02:46:51.238190 26284 raft_consensus.cc:2229] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:46:51.238330 26284 raft_consensus.cc:2258] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:51.238768 26282 ts_tablet_manager.cc:1905] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:51.239182 26284 ts_tablet_manager.cc:1905] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:51.241369 26282 ts_tablet_manager.cc:1918] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:46:51.241439 26282 log.cc:1196] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/6db2567568d44aa8a6dd475296e79b95
I20260811 02:46:51.241493 26285 ts_tablet_manager.cc:1905] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:51.241806 26282 ts_tablet_manager.cc:1939] T 6db2567568d44aa8a6dd475296e79b95 P 6dcfaed1cc6b40af93594457f7c760b1: Deleting consensus metadata
I20260811 02:46:51.242314 26284 ts_tablet_manager.cc:1918] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:46:51.242429 26284 log.cc:1196] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/6db2567568d44aa8a6dd475296e79b95
I20260811 02:46:51.242609 25292 catalog_manager.cc:4841] TS 6dcfaed1cc6b40af93594457f7c760b1 (127.21.236.130:44809): tablet 6db2567568d44aa8a6dd475296e79b95 (table test-table [id=2992d193f5af4d12b7c4a23773059745]) successfully deleted
I20260811 02:46:51.242786 26284 ts_tablet_manager.cc:1939] T 6db2567568d44aa8a6dd475296e79b95 P 04a8a7a4ee3c46019005cd78da00a953: Deleting consensus metadata
I20260811 02:46:51.243618 25293 catalog_manager.cc:4841] TS 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909): tablet 6db2567568d44aa8a6dd475296e79b95 (table test-table [id=2992d193f5af4d12b7c4a23773059745]) successfully deleted
I20260811 02:46:51.244184 26285 ts_tablet_manager.cc:1918] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:46:51.244277 26285 log.cc:1196] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/wal/wals/6db2567568d44aa8a6dd475296e79b95
I20260811 02:46:51.244550 26285 ts_tablet_manager.cc:1939] T 6db2567568d44aa8a6dd475296e79b95 P 74fea82c89ac488992fdec514870cde7: Deleting consensus metadata
I20260811 02:46:51.245580 25291 catalog_manager.cc:4841] TS 74fea82c89ac488992fdec514870cde7 (127.21.236.129:40691): tablet 6db2567568d44aa8a6dd475296e79b95 (table test-table [id=2992d193f5af4d12b7c4a23773059745]) successfully deleted
Aug 11 02:46:51 dist-test-slave-n326 krb5kdc[25249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416411, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:46:51 dist-test-slave-n326 krb5kdc[25249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416411, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:51.278045 26301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:51.260747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37698 (local address 127.21.236.190:33403)
0811 02:46:51.261064 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:51.261071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:51.261089 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:46:51.261283 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:51.261287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:51.261345 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:51.261527 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:51.262416 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.263132 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.264024 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.264259 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.264356 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:51.274200 (+ 9844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:51.274204 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:51.274206 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:51.274246 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:51.275970 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:51.276591 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:51.276596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:51.276597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:51.276704 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:51.277031 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:51.277034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:51.277036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:51.277261 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:51.277416 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:51.277756 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 02:46:51.277885 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":129,"threads_started":1}
I20260811 02:46:51.281082 25320 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:46:51.281524 25320 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:46:51.286326 25542 tablet_service.cc:1467] Processing CreateTablet for tablet 594a83f41a2a412e94b148455246792e (DEFAULT_TABLE table=test-table [id=e69055f7a3dc4eae8d027cbd68946bba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:51.286469 25772 tablet_service.cc:1467] Processing CreateTablet for tablet 594a83f41a2a412e94b148455246792e (DEFAULT_TABLE table=test-table [id=e69055f7a3dc4eae8d027cbd68946bba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:51.286540 26005 tablet_service.cc:1467] Processing CreateTablet for tablet 594a83f41a2a412e94b148455246792e (DEFAULT_TABLE table=test-table [id=e69055f7a3dc4eae8d027cbd68946bba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:51.286724 25542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 594a83f41a2a412e94b148455246792e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:51.286773 25772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 594a83f41a2a412e94b148455246792e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:51.286782 26005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 594a83f41a2a412e94b148455246792e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:51.289440 26302 tablet_bootstrap.cc:492] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7: Bootstrap starting.
I20260811 02:46:51.289491 26306 tablet_bootstrap.cc:492] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1: Bootstrap starting.
I20260811 02:46:51.289667 26305 tablet_bootstrap.cc:492] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953: Bootstrap starting.
I20260811 02:46:51.290361 26302 tablet_bootstrap.cc:654] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:51.290601 26305 tablet_bootstrap.cc:654] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:51.290738 26306 tablet_bootstrap.cc:654] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:51.291746 26302 tablet_bootstrap.cc:492] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7: No bootstrap required, opened a new log
I20260811 02:46:51.291858 26302 ts_tablet_manager.cc:1397] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 02:46:51.292040 26306 tablet_bootstrap.cc:492] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1: No bootstrap required, opened a new log
I20260811 02:46:51.292138 26306 ts_tablet_manager.cc:1397] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 02:46:51.292330 26302 raft_consensus.cc:348] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.292454 26302 raft_consensus.cc:374] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:51.292497 26302 raft_consensus.cc:729] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74fea82c89ac488992fdec514870cde7, State: Initialized, Role: FOLLOWER
I20260811 02:46:51.292615 26306 raft_consensus.cc:348] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.292646 26302 consensus_queue.cc:260] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.292722 26306 raft_consensus.cc:374] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:51.292747 26306 raft_consensus.cc:729] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dcfaed1cc6b40af93594457f7c760b1, State: Initialized, Role: FOLLOWER
I20260811 02:46:51.292888 26306 consensus_queue.cc:260] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.292969 26302 ts_tablet_manager.cc:1428] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:46:51.293565 26306 ts_tablet_manager.cc:1428] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 02:46:51.294620 26305 tablet_bootstrap.cc:492] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953: No bootstrap required, opened a new log
I20260811 02:46:51.294719 26305 ts_tablet_manager.cc:1397] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 02:46:51.295238 26305 raft_consensus.cc:348] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.295353 26305 raft_consensus.cc:374] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:51.295393 26305 raft_consensus.cc:729] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a8a7a4ee3c46019005cd78da00a953, State: Initialized, Role: FOLLOWER
I20260811 02:46:51.295574 26305 consensus_queue.cc:260] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.296020 26305 ts_tablet_manager.cc:1428] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 02:46:51.314095 26314 raft_consensus.cc:482] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:46:51.314245 26314 raft_consensus.cc:504] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.315570 26314 leader_election.cc:288] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74fea82c89ac488992fdec514870cde7 (127.21.236.129:40691), 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909)
I20260811 02:46:51.320096 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:51.316046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:54023 (local address 127.21.236.131:33909)
0811 02:46:51.316357 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:51.316370 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:51.316388 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:46:51.316451 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:51.316454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:51.316510 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:51.316616 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:51.317150 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.317831 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.318973 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.319494 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.319633 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:51.319692 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:51.319865 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 02:46:51.319939 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":113,"threads_started":1}
I20260811 02:46:51.320109 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:51.315788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:33909 (local address 127.21.236.130:54023)
0811 02:46:51.316194 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:51.316212 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:51.316319 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:51.316798 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:51.316801 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:51.316844 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:51.317003 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:51.317010 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.317958 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.317962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:51.318841 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:51.318850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.319630 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.319634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:51.319732 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:51.319759 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 02:46:51.319845 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 02:46:51.319919 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":59,"spinlock_wait_cycles":768,"thread_start_us":146,"threads_started":1}
I20260811 02:46:51.320183 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:51.315788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:40691 (local address 127.21.236.130:43939)
0811 02:46:51.316319 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:51.316335 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:51.316474 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:51.316761 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:51.316765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:51.316786 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:46:51.316978 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:51.316985 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.317999 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.318003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:51.318878 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:51.318888 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.319688 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.319692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:51.319801 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:51.319827 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 02:46:51.319920 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 02:46:51.319997 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":28,"spinlock_wait_cycles":23296,"thread_start_us":147,"threads_started":1}
I20260811 02:46:51.320641 26038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "594a83f41a2a412e94b148455246792e" candidate_uuid: "6dcfaed1cc6b40af93594457f7c760b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a8a7a4ee3c46019005cd78da00a953" is_pre_election: true
I20260811 02:46:51.320724 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:51.315915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:43939 (local address 127.21.236.129:40691)
0811 02:46:51.316216 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:51.316226 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:51.316459 (+ 233us) server_negotiation.cc:407] Connection header received
0811 02:46:51.316563 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:51.316565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:51.316633 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:51.316722 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:46:51.317109 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.317852 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.319013 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.319559 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.319698 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:51.319751 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:51.319956 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 02:46:51.320041 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":132,"threads_started":1}
I20260811 02:46:51.320820 26038 raft_consensus.cc:2454] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dcfaed1cc6b40af93594457f7c760b1 in term 0.
I20260811 02:46:51.321215 25568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "594a83f41a2a412e94b148455246792e" candidate_uuid: "6dcfaed1cc6b40af93594457f7c760b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fea82c89ac488992fdec514870cde7" is_pre_election: true
I20260811 02:46:51.321254 25693 leader_election.cc:302] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a8a7a4ee3c46019005cd78da00a953, 6dcfaed1cc6b40af93594457f7c760b1; no voters:
I20260811 02:46:51.321481 25568 raft_consensus.cc:2454] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dcfaed1cc6b40af93594457f7c760b1 in term 0.
I20260811 02:46:51.321612 26314 raft_consensus.cc:2781] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:46:51.321696 26314 raft_consensus.cc:482] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:46:51.321755 26314 raft_consensus.cc:3037] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:51.322872 26314 raft_consensus.cc:504] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.323318 26314 leader_election.cc:288] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [CANDIDATE]: Term 1 election: Requested vote from peers 74fea82c89ac488992fdec514870cde7 (127.21.236.129:40691), 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909)
I20260811 02:46:51.323755 25568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "594a83f41a2a412e94b148455246792e" candidate_uuid: "6dcfaed1cc6b40af93594457f7c760b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fea82c89ac488992fdec514870cde7"
I20260811 02:46:51.323767 26038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "594a83f41a2a412e94b148455246792e" candidate_uuid: "6dcfaed1cc6b40af93594457f7c760b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a8a7a4ee3c46019005cd78da00a953"
I20260811 02:46:51.323889 26038 raft_consensus.cc:3037] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:51.323889 25568 raft_consensus.cc:3037] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:51.325050 26038 raft_consensus.cc:2454] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dcfaed1cc6b40af93594457f7c760b1 in term 1.
I20260811 02:46:51.325158 25568 raft_consensus.cc:2454] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dcfaed1cc6b40af93594457f7c760b1 in term 1.
I20260811 02:46:51.325408 25693 leader_election.cc:302] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a8a7a4ee3c46019005cd78da00a953, 6dcfaed1cc6b40af93594457f7c760b1; no voters:
I20260811 02:46:51.325661 26314 raft_consensus.cc:2781] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:51.325924 26314 raft_consensus.cc:686] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 LEADER]: Becoming Leader. State: Replica: 6dcfaed1cc6b40af93594457f7c760b1, State: Running, Role: LEADER
I20260811 02:46:51.326231 26314 consensus_queue.cc:237] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.328609 25316 catalog_manager.cc:5495] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dcfaed1cc6b40af93594457f7c760b1 (127.21.236.130). New cstate: current_term: 1 leader_uuid: "6dcfaed1cc6b40af93594457f7c760b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260811 02:46:51.351994 26331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:51.347226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55286 (local address 127.21.236.130:44809)
0811 02:46:51.347568 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:51.347575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:51.347587 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:46:51.347668 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:51.347671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:51.347726 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:51.347830 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:46:51.348069 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.348896 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.350174 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.350459 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.350583 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:51.350838 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:46:51.351411 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:46:51.351527 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:51.351806 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 02:46:51.351861 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":177,"threads_started":1}
I20260811 02:46:51.363214 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:51.359623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:60711 (local address 127.21.236.129:40691)
0811 02:46:51.359773 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:51.359781 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:51.360046 (+ 265us) server_negotiation.cc:407] Connection header received
0811 02:46:51.360138 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:51.360140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:51.360191 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:51.360257 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:46:51.360615 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.361165 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.362103 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.362335 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.362422 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:51.362722 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:46:51.362802 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:46:51.362884 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:51.362962 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 02:46:51.363015 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:46:51.363233 26337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:51.359522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:40691 (local address 127.21.236.130:60711)
0811 02:46:51.359946 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:51.359959 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:51.360053 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:51.360313 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:51.360315 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:51.360323 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:46:51.360440 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:51.360445 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.361284 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.361286 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:51.361966 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:51.361976 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.362442 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.362444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:51.362514 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:51.362561 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:46:51.362890 (+ 329us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:46:51.362894 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 02:46:51.362940 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 02:46:51.363006 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":122,"threads_started":1}
I20260811 02:46:51.369365 26038 raft_consensus.cc:1261] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Refusing update from remote peer 6dcfaed1cc6b40af93594457f7c760b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:51.369634 25568 raft_consensus.cc:1261] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7 [term 1 FOLLOWER]: Refusing update from remote peer 6dcfaed1cc6b40af93594457f7c760b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:51.369942 26324 consensus_queue.cc:1035] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:46:51.370174 26314 consensus_queue.cc:1035] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:46:51.388419 26331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:51.384897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:37921 (local address 127.21.236.130:44809)
0811 02:46:51.385020 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:51.385025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:51.385276 (+ 251us) server_negotiation.cc:407] Connection header received
0811 02:46:51.385403 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:51.385405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:51.385453 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:51.385538 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:46:51.385792 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.386441 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.387365 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.387608 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.387716 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:51.387965 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:46:51.388046 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:46:51.388124 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:51.388221 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 02:46:51.388276 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 02:46:51.388424 26342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:51.384852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:44809 (local address 127.21.236.129:37921)
0811 02:46:51.385182 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:51.385194 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:51.385317 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:51.385548 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:51.385551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:51.385558 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:46:51.385679 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:51.385685 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.386546 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.386551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:51.387246 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:51.387257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:51.387719 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:51.387723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:51.387812 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:51.387848 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:46:51.388127 (+ 279us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:46:51.388136 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 02:46:51.388199 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 02:46:51.388282 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":110,"threads_started":1}
I20260811 02:46:51.414476 25316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:46:51.414659 25316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:46:51.416953 25316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9: Sending DeleteTablet for 3 replicas of tablet 594a83f41a2a412e94b148455246792e
I20260811 02:46:51.417598 25772 tablet_service.cc:1514] Processing DeleteTablet for tablet 594a83f41a2a412e94b148455246792e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:51 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:55280
I20260811 02:46:51.417603 25542 tablet_service.cc:1514] Processing DeleteTablet for tablet 594a83f41a2a412e94b148455246792e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:51 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:58338
I20260811 02:46:51.417809 26285 tablet_replica.cc:330] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7: stopping tablet replica
I20260811 02:46:51.417832 26005 tablet_service.cc:1514] Processing DeleteTablet for tablet 594a83f41a2a412e94b148455246792e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:51 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:42938
I20260811 02:46:51.417943 26285 raft_consensus.cc:2229] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:46:51.418066 26285 raft_consensus.cc:2258] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:51.418237 26282 tablet_replica.cc:330] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1: stopping tablet replica
I20260811 02:46:51.418358 26284 tablet_replica.cc:330] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953: stopping tablet replica
I20260811 02:46:51.418355 26282 raft_consensus.cc:2229] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:46:51.418465 26284 raft_consensus.cc:2229] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:46:51.418578 26284 raft_consensus.cc:2258] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:51.418718 26282 raft_consensus.cc:2258] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:51.419296 25316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:46:51.419515 26284 ts_tablet_manager.cc:1905] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:51.419711 26282 ts_tablet_manager.cc:1905] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 02:46:51.420040 25316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:46:51.421516 26285 ts_tablet_manager.cc:1905] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:51.422441 26282 ts_tablet_manager.cc:1918] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:46:51.422515 26282 log.cc:1196] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/594a83f41a2a412e94b148455246792e
I20260811 02:46:51.423045 26284 ts_tablet_manager.cc:1918] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:46:51.423159 26284 log.cc:1196] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/594a83f41a2a412e94b148455246792e
I20260811 02:46:51.423488 26284 ts_tablet_manager.cc:1939] T 594a83f41a2a412e94b148455246792e P 04a8a7a4ee3c46019005cd78da00a953: Deleting consensus metadata
I20260811 02:46:51.424233 26282 ts_tablet_manager.cc:1939] T 594a83f41a2a412e94b148455246792e P 6dcfaed1cc6b40af93594457f7c760b1: Deleting consensus metadata
I20260811 02:46:51.424254 25293 catalog_manager.cc:4841] TS 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909): tablet 594a83f41a2a412e94b148455246792e (table test-table [id=e69055f7a3dc4eae8d027cbd68946bba]) successfully deleted
I20260811 02:46:51.425105 25292 catalog_manager.cc:4841] TS 6dcfaed1cc6b40af93594457f7c760b1 (127.21.236.130:44809): tablet 594a83f41a2a412e94b148455246792e (table test-table [id=e69055f7a3dc4eae8d027cbd68946bba]) successfully deleted
I20260811 02:46:51.425166 26285 ts_tablet_manager.cc:1918] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:46:51.425352 26285 log.cc:1196] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-0/wal/wals/594a83f41a2a412e94b148455246792e
I20260811 02:46:51.425710 26285 ts_tablet_manager.cc:1939] T 594a83f41a2a412e94b148455246792e P 74fea82c89ac488992fdec514870cde7: Deleting consensus metadata
I20260811 02:46:51.426515 25291 catalog_manager.cc:4841] TS 74fea82c89ac488992fdec514870cde7 (127.21.236.129:40691): tablet 594a83f41a2a412e94b148455246792e (table test-table [id=e69055f7a3dc4eae8d027cbd68946bba]) successfully deleted
I20260811 02:46:51.428529 25542 tablet_service.cc:1467] Processing CreateTablet for tablet ddefe7c0fe204dd2a7e6299452564842 (DEFAULT_TABLE table=test-table [id=f5581769a51d4bd2ac373629ec6a058e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:51.428817 25542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddefe7c0fe204dd2a7e6299452564842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:51.428830 26005 tablet_service.cc:1467] Processing CreateTablet for tablet ddefe7c0fe204dd2a7e6299452564842 (DEFAULT_TABLE table=test-table [id=f5581769a51d4bd2ac373629ec6a058e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:51.428830 25772 tablet_service.cc:1467] Processing CreateTablet for tablet ddefe7c0fe204dd2a7e6299452564842 (DEFAULT_TABLE table=test-table [id=f5581769a51d4bd2ac373629ec6a058e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:46:51.429114 26005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddefe7c0fe204dd2a7e6299452564842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:51.429109 25772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddefe7c0fe204dd2a7e6299452564842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:51.431375 26302 tablet_bootstrap.cc:492] T ddefe7c0fe204dd2a7e6299452564842 P 74fea82c89ac488992fdec514870cde7: Bootstrap starting.
I20260811 02:46:51.431636 26306 tablet_bootstrap.cc:492] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1: Bootstrap starting.
I20260811 02:46:51.431978 26305 tablet_bootstrap.cc:492] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953: Bootstrap starting.
I20260811 02:46:51.432255 26302 tablet_bootstrap.cc:654] T ddefe7c0fe204dd2a7e6299452564842 P 74fea82c89ac488992fdec514870cde7: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:51.432714 26306 tablet_bootstrap.cc:654] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:51.433077 26305 tablet_bootstrap.cc:654] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:51.434621 26302 tablet_bootstrap.cc:492] T ddefe7c0fe204dd2a7e6299452564842 P 74fea82c89ac488992fdec514870cde7: No bootstrap required, opened a new log
I20260811 02:46:51.434710 26302 ts_tablet_manager.cc:1397] T ddefe7c0fe204dd2a7e6299452564842 P 74fea82c89ac488992fdec514870cde7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 02:46:51.434799 26306 tablet_bootstrap.cc:492] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1: No bootstrap required, opened a new log
I20260811 02:46:51.434890 26306 ts_tablet_manager.cc:1397] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 02:46:51.435003 26305 tablet_bootstrap.cc:492] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953: No bootstrap required, opened a new log
I20260811 02:46:51.435117 26305 ts_tablet_manager.cc:1397] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:46:51.435145 26302 raft_consensus.cc:348] T ddefe7c0fe204dd2a7e6299452564842 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.435271 26302 raft_consensus.cc:374] T ddefe7c0fe204dd2a7e6299452564842 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:51.435302 26302 raft_consensus.cc:729] T ddefe7c0fe204dd2a7e6299452564842 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74fea82c89ac488992fdec514870cde7, State: Initialized, Role: FOLLOWER
I20260811 02:46:51.435424 26302 consensus_queue.cc:260] T ddefe7c0fe204dd2a7e6299452564842 P 74fea82c89ac488992fdec514870cde7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.435446 26306 raft_consensus.cc:348] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.435580 26306 raft_consensus.cc:374] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:51.435627 26306 raft_consensus.cc:729] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dcfaed1cc6b40af93594457f7c760b1, State: Initialized, Role: FOLLOWER
I20260811 02:46:51.435561 26305 raft_consensus.cc:348] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.435683 26302 ts_tablet_manager.cc:1428] T ddefe7c0fe204dd2a7e6299452564842 P 74fea82c89ac488992fdec514870cde7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 02:46:51.435696 26305 raft_consensus.cc:374] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:51.435735 26305 raft_consensus.cc:729] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04a8a7a4ee3c46019005cd78da00a953, State: Initialized, Role: FOLLOWER
I20260811 02:46:51.435766 26306 consensus_queue.cc:260] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.435889 26305 consensus_queue.cc:260] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.436060 26306 ts_tablet_manager.cc:1428] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:46:51.436175 26305 ts_tablet_manager.cc:1428] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:46:51.908087 26324 raft_consensus.cc:482] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:46:51.908298 26324 raft_consensus.cc:504] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.908829 26324 leader_election.cc:288] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74fea82c89ac488992fdec514870cde7 (127.21.236.129:40691), 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909)
I20260811 02:46:51.909408 25568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddefe7c0fe204dd2a7e6299452564842" candidate_uuid: "6dcfaed1cc6b40af93594457f7c760b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fea82c89ac488992fdec514870cde7" is_pre_election: true
I20260811 02:46:51.909461 26038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddefe7c0fe204dd2a7e6299452564842" candidate_uuid: "6dcfaed1cc6b40af93594457f7c760b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a8a7a4ee3c46019005cd78da00a953" is_pre_election: true
I20260811 02:46:51.909631 26038 raft_consensus.cc:2454] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dcfaed1cc6b40af93594457f7c760b1 in term 0.
I20260811 02:46:51.909641 25568 raft_consensus.cc:2454] T ddefe7c0fe204dd2a7e6299452564842 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6dcfaed1cc6b40af93594457f7c760b1 in term 0.
I20260811 02:46:51.909955 25693 leader_election.cc:302] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04a8a7a4ee3c46019005cd78da00a953, 6dcfaed1cc6b40af93594457f7c760b1; no voters:
I20260811 02:46:51.910151 26324 raft_consensus.cc:2781] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:46:51.910219 26324 raft_consensus.cc:482] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:46:51.910267 26324 raft_consensus.cc:3037] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:51.911554 26324 raft_consensus.cc:504] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.911934 26324 leader_election.cc:288] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [CANDIDATE]: Term 1 election: Requested vote from peers 74fea82c89ac488992fdec514870cde7 (127.21.236.129:40691), 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909)
I20260811 02:46:51.912348 25568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddefe7c0fe204dd2a7e6299452564842" candidate_uuid: "6dcfaed1cc6b40af93594457f7c760b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fea82c89ac488992fdec514870cde7"
I20260811 02:46:51.912351 26038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddefe7c0fe204dd2a7e6299452564842" candidate_uuid: "6dcfaed1cc6b40af93594457f7c760b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04a8a7a4ee3c46019005cd78da00a953"
I20260811 02:46:51.912462 25568 raft_consensus.cc:3037] T ddefe7c0fe204dd2a7e6299452564842 P 74fea82c89ac488992fdec514870cde7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:51.912488 26038 raft_consensus.cc:3037] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:51.913481 25568 raft_consensus.cc:2454] T ddefe7c0fe204dd2a7e6299452564842 P 74fea82c89ac488992fdec514870cde7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dcfaed1cc6b40af93594457f7c760b1 in term 1.
I20260811 02:46:51.913605 26038 raft_consensus.cc:2454] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6dcfaed1cc6b40af93594457f7c760b1 in term 1.
I20260811 02:46:51.913848 25690 leader_election.cc:302] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6dcfaed1cc6b40af93594457f7c760b1, 74fea82c89ac488992fdec514870cde7; no voters:
I20260811 02:46:51.914088 26324 raft_consensus.cc:2781] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:51.914219 26324 raft_consensus.cc:686] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 LEADER]: Becoming Leader. State: Replica: 6dcfaed1cc6b40af93594457f7c760b1, State: Running, Role: LEADER
I20260811 02:46:51.914374 26324 consensus_queue.cc:237] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } }
I20260811 02:46:51.916035 25310 catalog_manager.cc:5495] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6dcfaed1cc6b40af93594457f7c760b1 (127.21.236.130). New cstate: current_term: 1 leader_uuid: "6dcfaed1cc6b40af93594457f7c760b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6dcfaed1cc6b40af93594457f7c760b1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 } health_report { overall_health: UNKNOWN } } }
I20260811 02:46:52.011647 25568 raft_consensus.cc:1261] T ddefe7c0fe204dd2a7e6299452564842 P 74fea82c89ac488992fdec514870cde7 [term 1 FOLLOWER]: Refusing update from remote peer 6dcfaed1cc6b40af93594457f7c760b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:52.011637 26038 raft_consensus.cc:1261] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Refusing update from remote peer 6dcfaed1cc6b40af93594457f7c760b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:46:52.012029 26324 consensus_queue.cc:1035] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74fea82c89ac488992fdec514870cde7" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:46:52.012192 26347 consensus_queue.cc:1035] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04a8a7a4ee3c46019005cd78da00a953" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 02:46:52.025888 26356 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 02:46:53.589720 25316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:46:53.589881 25316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:46:53.592484 25316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8d8bca6382f45dfad7fa09504f694d9: Sending DeleteTablet for 3 replicas of tablet ddefe7c0fe204dd2a7e6299452564842
I20260811 02:46:53.593113 25542 tablet_service.cc:1514] Processing DeleteTablet for tablet ddefe7c0fe204dd2a7e6299452564842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:53 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:58338
I20260811 02:46:53.593132 25772 tablet_service.cc:1514] Processing DeleteTablet for tablet ddefe7c0fe204dd2a7e6299452564842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:53 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:55280
I20260811 02:46:53.593223 26005 tablet_service.cc:1514] Processing DeleteTablet for tablet ddefe7c0fe204dd2a7e6299452564842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:46:53 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:42938
I20260811 02:46:53.593597 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 25430
I20260811 02:46:53.593607 26430 tablet_replica.cc:330] T ddefe7c0fe204dd2a7e6299452564842 P 74fea82c89ac488992fdec514870cde7: stopping tablet replica
I20260811 02:46:53.593606 26432 tablet_replica.cc:330] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953: stopping tablet replica
I20260811 02:46:53.609026 26432 raft_consensus.cc:2229] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:46:53.609334 26432 raft_consensus.cc:2258] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:53.610334 26432 ts_tablet_manager.cc:1905] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:53.610770 26431 tablet_replica.cc:330] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1: stopping tablet replica
I20260811 02:46:53.611191 26431 raft_consensus.cc:2229] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:46:53.611608 26431 raft_consensus.cc:2258] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:46:53.613497 26431 ts_tablet_manager.cc:1905] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:46:53.613648 26432 ts_tablet_manager.cc:1918] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:46:53.613795 26432 log.cc:1196] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/ddefe7c0fe204dd2a7e6299452564842
I20260811 02:46:53.614167 26432 ts_tablet_manager.cc:1939] T ddefe7c0fe204dd2a7e6299452564842 P 04a8a7a4ee3c46019005cd78da00a953: Deleting consensus metadata
I20260811 02:46:53.617669 26431 ts_tablet_manager.cc:1918] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 02:46:53.617835 26431 log.cc:1196] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/ddefe7c0fe204dd2a7e6299452564842
I20260811 02:46:53.618268 26431 ts_tablet_manager.cc:1939] T ddefe7c0fe204dd2a7e6299452564842 P 6dcfaed1cc6b40af93594457f7c760b1: Deleting consensus metadata
I20260811 02:46:53.619130 25292 catalog_manager.cc:4841] TS 6dcfaed1cc6b40af93594457f7c760b1 (127.21.236.130:44809): tablet ddefe7c0fe204dd2a7e6299452564842 (table test-table [id=f5581769a51d4bd2ac373629ec6a058e]) successfully deleted
I20260811 02:46:53.630999 25293 catalog_manager.cc:4841] TS 04a8a7a4ee3c46019005cd78da00a953 (127.21.236.131:33909): tablet ddefe7c0fe204dd2a7e6299452564842 (table test-table [id=f5581769a51d4bd2ac373629ec6a058e]) successfully deleted
W20260811 02:46:53.648089 25291 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:40691 (error 108)
I20260811 02:46:53.648993 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 25666
I20260811 02:46:53.649808 26436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:53.648941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:40691 (local address 127.0.0.1:58374)
0811 02:46:53.649233 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:53.649322 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:40691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":101,"threads_started":1}
W20260811 02:46:53.651942 25291 catalog_manager.cc:4584] TS 74fea82c89ac488992fdec514870cde7 (127.21.236.129:40691): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ddefe7c0fe204dd2a7e6299452564842: Network error: Client connection negotiation failed: client connection to 127.21.236.129:40691: connect: Connection refused (error 111)
I20260811 02:46:53.665968 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 25889
I20260811 02:46:53.681051 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 25268
2026-08-11T02:46:53Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7690 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:46:53 dist-test-slave-n326 krb5kdc[26441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:46:53 dist-test-slave-n326 krb5kdc[26441](info): set up 2 sockets
Aug 11 02:46:53 dist-test-slave-n326 krb5kdc[26441](info): commencing operation
krb5kdc: starting...
W20260811 02:46:54.982478 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:46:55 dist-test-slave-n326 krb5kdc[26441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:46:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:46:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:46:55.118966 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:38227
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:44693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:38227
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:55.234401 26472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:55.234735 26472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:55.234808 26472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:55.238798 26472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:46:55.238901 26472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:55.238941 26472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:55.238978 26472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:46:55.239019 26472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:46:55.244383 26472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:44693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:38227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:38227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:55.245497 26472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:55.246549 26472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:55.255751 26480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:55.255777 26472 server_base.cc:1034] running on GCE node
W20260811 02:46:55.255936 26483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:55.256084 26485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:55.256647 26472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:55.257792 26472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:55.258976 26472 hybrid_clock.cc:648] HybridClock initialized: now 1786416415258947 us; error 41 us; skew 500 ppm
Aug 11 02:46:55 dist-test-slave-n326 krb5kdc[26441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416415, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:55.261936 26472 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:46:55.262977 26472 webserver.cc:460] Webserver started at http://127.21.236.190:45225/ using document root <none> and password file <none>
I20260811 02:46:55.263628 26472 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:55.263729 26472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:55.263979 26472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:55.266261 26472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "a87ec188f982451c91cf1608af5745fe"
format_stamp: "Formatted at 2026-08-11 02:46:55 on dist-test-slave-n326"
server_key: "35ab7075c61db32fa85f68083a4a9f61"
server_key_iv: "6339bc58fe3bbdac541e07bf3e19def5"
server_key_version: "encryptionkey@0"
I20260811 02:46:55.266880 26472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "a87ec188f982451c91cf1608af5745fe"
format_stamp: "Formatted at 2026-08-11 02:46:55 on dist-test-slave-n326"
server_key: "35ab7075c61db32fa85f68083a4a9f61"
server_key_iv: "6339bc58fe3bbdac541e07bf3e19def5"
server_key_version: "encryptionkey@0"
I20260811 02:46:55.271237 26472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:46:55.273941 26499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:55.275378 26472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:46:55.275669 26472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "a87ec188f982451c91cf1608af5745fe"
format_stamp: "Formatted at 2026-08-11 02:46:55 on dist-test-slave-n326"
server_key: "35ab7075c61db32fa85f68083a4a9f61"
server_key_iv: "6339bc58fe3bbdac541e07bf3e19def5"
server_key_version: "encryptionkey@0"
I20260811 02:46:55.275913 26472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:55.317586 26472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:55.320928 26472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:55.321168 26472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:55.328408 26472 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:38227
I20260811 02:46:55.328430 26588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:38227 every 8 connection(s)
I20260811 02:46:55.329404 26472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:46:55.332314 26589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:55.337307 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 26472
I20260811 02:46:55.337431 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:46:55.337720 22450 external_mini_cluster.cc:1428] Setting key 1f815a5fec379905827542221060b54b
I20260811 02:46:55.338351 26589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe: Bootstrap starting.
I20260811 02:46:55.340839 26589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:55.341828 26589 log.cc:824] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:55.344003 26589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe: No bootstrap required, opened a new log
Aug 11 02:46:55 dist-test-slave-n326 krb5kdc[26441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:55.348457 26589 raft_consensus.cc:348] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a87ec188f982451c91cf1608af5745fe" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38227 } }
I20260811 02:46:55.348794 26589 raft_consensus.cc:374] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:55.348882 26589 raft_consensus.cc:729] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a87ec188f982451c91cf1608af5745fe, State: Initialized, Role: FOLLOWER
I20260811 02:46:55.349380 26589 consensus_queue.cc:260] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a87ec188f982451c91cf1608af5745fe" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38227 } }
I20260811 02:46:55.349507 26589 raft_consensus.cc:388] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:46:55.349596 26589 raft_consensus.cc:482] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:46:55.349712 26589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:55.350641 26589 raft_consensus.cc:504] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a87ec188f982451c91cf1608af5745fe" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38227 } }
I20260811 02:46:55.350993 26589 leader_election.cc:302] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a87ec188f982451c91cf1608af5745fe; no voters:
I20260811 02:46:55.351384 26589 leader_election.cc:288] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:46:55.351457 26600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:55.351671 26600 raft_consensus.cc:686] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [term 1 LEADER]: Becoming Leader. State: Replica: a87ec188f982451c91cf1608af5745fe, State: Running, Role: LEADER
I20260811 02:46:55.352017 26600 consensus_queue.cc:237] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a87ec188f982451c91cf1608af5745fe" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38227 } }
I20260811 02:46:55.352783 26589 sys_catalog.cc:564] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:46:55.353211 26600 sys_catalog.cc:455] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader a87ec188f982451c91cf1608af5745fe. Latest consensus state: current_term: 1 leader_uuid: "a87ec188f982451c91cf1608af5745fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a87ec188f982451c91cf1608af5745fe" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38227 } } }
I20260811 02:46:55.353412 26600 sys_catalog.cc:458] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [sys.catalog]: This master's current role is: LEADER
I20260811 02:46:55.353607 26604 sys_catalog.cc:455] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a87ec188f982451c91cf1608af5745fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a87ec188f982451c91cf1608af5745fe" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38227 } } }
I20260811 02:46:55.353773 26604 sys_catalog.cc:458] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe [sys.catalog]: This master's current role is: LEADER
I20260811 02:46:55.353874 26608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:46:55.354141 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:55.339264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.21.236.190:38227)
0811 02:46:55.339741 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:55.339758 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:55.339796 (+ 38us) server_negotiation.cc:407] Connection header received
0811 02:46:55.340477 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:55.340495 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:55.340873 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:55.341236 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:55.342216 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:55.343375 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:55.344568 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:55.345073 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:55.345216 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:55.348183 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:55.348188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:55.348202 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:55.348229 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:55.350315 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:55.350770 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:55.350777 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:55.350783 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:55.350910 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:55.351237 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:55.351241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:55.351242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:55.351796 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:55.352601 (+ 805us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:55.352725 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 02:46:55.353023 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":159,"threads_started":1}
I20260811 02:46:55.357715 26608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:46:55.363796 26608 catalog_manager.cc:1347] Generated new cluster ID: 6df565cb6fba4c3cbd07f207ee1ae72d
I20260811 02:46:55.363873 26608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:46:55.373967 26608 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:46:55.375275 26608 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:46:55.383481 26608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a87ec188f982451c91cf1608af5745fe: Generated new TSK 0
I20260811 02:46:55.384142 26608 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:46:55.417852 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:38227
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:44693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:55.530478 26636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:55.530735 26636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:55.530777 26636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:55.534767 26636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:55.534827 26636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:55.534884 26636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:46:55.540079 26636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:44693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:38227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:55.541049 26636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:55.542080 26636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:55.551571 26644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:55.551601 26645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:55.551751 26648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:55.551739 26636 server_base.cc:1034] running on GCE node
I20260811 02:46:55.552235 26636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:55.552933 26636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:55.554093 26636 hybrid_clock.cc:648] HybridClock initialized: now 1786416415554077 us; error 26 us; skew 500 ppm
Aug 11 02:46:55 dist-test-slave-n326 krb5kdc[26441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416415, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:55.556928 26636 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:46:55.557812 26636 webserver.cc:460] Webserver started at http://127.21.236.129:44663/ using document root <none> and password file <none>
I20260811 02:46:55.558378 26636 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:55.558470 26636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:55.558706 26636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:55.560966 26636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "da950689fbf74fb5b992f3f413af54d0"
format_stamp: "Formatted at 2026-08-11 02:46:55 on dist-test-slave-n326"
server_key: "43f0baa47057031b333547401c3103dd"
server_key_iv: "4f9fcf8604e3b33242232b28d94e59c4"
server_key_version: "encryptionkey@0"
I20260811 02:46:55.561647 26636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "da950689fbf74fb5b992f3f413af54d0"
format_stamp: "Formatted at 2026-08-11 02:46:55 on dist-test-slave-n326"
server_key: "43f0baa47057031b333547401c3103dd"
server_key_iv: "4f9fcf8604e3b33242232b28d94e59c4"
server_key_version: "encryptionkey@0"
I20260811 02:46:55.565737 26636 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:46:55.568413 26655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:55.569767 26636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:46:55.569916 26636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "da950689fbf74fb5b992f3f413af54d0"
format_stamp: "Formatted at 2026-08-11 02:46:55 on dist-test-slave-n326"
server_key: "43f0baa47057031b333547401c3103dd"
server_key_iv: "4f9fcf8604e3b33242232b28d94e59c4"
server_key_version: "encryptionkey@0"
I20260811 02:46:55.570044 26636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:55.587267 26636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:55.590657 26636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:55.590868 26636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:55.591539 26636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:46:55.592379 26636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:55.592453 26636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:55.592517 26636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:55.592548 26636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:55.601553 26636 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:33333
I20260811 02:46:55.601590 26847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:33333 every 8 connection(s)
I20260811 02:46:55.602473 26636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:46:55.604676 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 26636
I20260811 02:46:55.604799 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:46:55.605082 22450 external_mini_cluster.cc:1428] Setting key 69da908e5a7d2931191f6d6a361b29f7
Aug 11 02:46:55 dist-test-slave-n326 krb5kdc[26441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416415, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
I20260811 02:46:55.617534 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:55.604336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:36761 (local address 127.21.236.190:38227)
0811 02:46:55.604583 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:55.604588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:55.605462 (+ 874us) server_negotiation.cc:407] Connection header received
0811 02:46:55.607041 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:55.607045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:55.607144 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:55.607222 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:55.608383 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:55.608925 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:55.610066 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:55.610311 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:55.610420 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:55.613548 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:55.613551 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:55.613554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:55.613578 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:55.615014 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:55.615737 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:55.615740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:55.615742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:55.615883 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:55.616314 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:55.616318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:55.616320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:55.616540 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:55.616734 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:55.617265 (+ 531us) server_negotiation.cc:299] Negotiation successful
0811 02:46:55.617408 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 02:46:55.618371 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:55.604765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38227 (local address 127.21.236.129:36761)
0811 02:46:55.605294 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:55.605347 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:55.606833 (+ 1486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:55.607395 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:55.607402 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:55.607759 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:55.608239 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:55.608252 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:55.609059 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:55.609063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:55.609940 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:55.609950 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:55.610470 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:55.610475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:55.610672 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:55.611368 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:55.611392 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:55.613368 (+ 1976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:55.615176 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:55.615184 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:55.615203 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:55.615608 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:55.616018 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:55.616023 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:55.616025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:55.616194 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:55.616743 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:55.616749 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:55.616993 (+ 244us) client_negotiation.cc:769] Sending connection context
0811 02:46:55.617186 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 02:46:55.617473 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":140,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
I20260811 02:46:55.619767 26849 heartbeater.cc:347] Connected to a master server at 127.21.236.190:38227
I20260811 02:46:55.620035 26849 heartbeater.cc:464] Registering TS with master...
I20260811 02:46:55.620569 26849 heartbeater.cc:511] Master 127.21.236.190:38227 requested a full tablet report, sending...
I20260811 02:46:55.622360 26533 ts_manager.cc:194] Registered new tserver with Master: da950689fbf74fb5b992f3f413af54d0 (127.21.236.129:33333)
I20260811 02:46:55.623605 26533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:36761
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:46:55.634665 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:38227
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:44693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:55.752573 26861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:55.752992 26861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:55.753060 26861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:55.757265 26861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:55.757333 26861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:55.757396 26861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:46:55.762988 26861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:44693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:38227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:55.764151 26861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:55.765177 26861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:55.775537 26870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:55.775590 26874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:55.775538 26871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:55.775835 26861 server_base.cc:1034] running on GCE node
I20260811 02:46:55.776330 26861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:55.777070 26861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:55.778276 26861 hybrid_clock.cc:648] HybridClock initialized: now 1786416415778256 us; error 29 us; skew 500 ppm
Aug 11 02:46:55 dist-test-slave-n326 krb5kdc[26441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416415, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:55.781752 26861 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:46:55.782765 26861 webserver.cc:460] Webserver started at http://127.21.236.130:41123/ using document root <none> and password file <none>
I20260811 02:46:55.783492 26861 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:55.783563 26861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:55.783747 26861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:55.785976 26861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "3ed67c4e1025471cb8a614960e493274"
format_stamp: "Formatted at 2026-08-11 02:46:55 on dist-test-slave-n326"
server_key: "9a45705efa1710b5f4de36ef64ddae19"
server_key_iv: "1e1c8606e1bee9441b1d37da7936ef6d"
server_key_version: "encryptionkey@0"
I20260811 02:46:55.786537 26861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "3ed67c4e1025471cb8a614960e493274"
format_stamp: "Formatted at 2026-08-11 02:46:55 on dist-test-slave-n326"
server_key: "9a45705efa1710b5f4de36ef64ddae19"
server_key_iv: "1e1c8606e1bee9441b1d37da7936ef6d"
server_key_version: "encryptionkey@0"
I20260811 02:46:55.790618 26861 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:46:55.793596 26887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:55.795019 26861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:46:55.795219 26861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "3ed67c4e1025471cb8a614960e493274"
format_stamp: "Formatted at 2026-08-11 02:46:55 on dist-test-slave-n326"
server_key: "9a45705efa1710b5f4de36ef64ddae19"
server_key_iv: "1e1c8606e1bee9441b1d37da7936ef6d"
server_key_version: "encryptionkey@0"
I20260811 02:46:55.795333 26861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:55.843011 26861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:55.846714 26861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:55.846961 26861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:55.847700 26861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:46:55.848646 26861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:55.848699 26861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:55.848771 26861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:55.848810 26861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:55.858970 26861 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:38175
I20260811 02:46:55.858987 27090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:38175 every 8 connection(s)
I20260811 02:46:55.860001 26861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:46:55.864234 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 26861
I20260811 02:46:55.864362 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:46:55.864624 22450 external_mini_cluster.cc:1428] Setting key b06f5a74d03d3a9fdef41cc54ef78433
Aug 11 02:46:55 dist-test-slave-n326 krb5kdc[26441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416415, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:55.875397 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:55.861946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:50275 (local address 127.21.236.190:38227)
0811 02:46:55.862091 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:55.862095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:55.862929 (+ 834us) server_negotiation.cc:407] Connection header received
0811 02:46:55.864549 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:55.864553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:55.864618 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:55.864733 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:55.866187 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:55.866766 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:55.868089 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:55.868336 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:55.868461 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:55.871546 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:55.871549 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:55.871552 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:55.871576 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:55.873128 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:55.873727 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:55.873731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:55.873738 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:55.873813 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:55.874186 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:55.874189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:55.874191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:55.874446 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:55.874551 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:55.875053 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 02:46:55.875213 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
I20260811 02:46:55.876223 27096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:55.862314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38227 (local address 127.21.236.130:50275)
0811 02:46:55.862767 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:55.862818 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:55.864292 (+ 1474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:55.864934 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:55.864941 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:55.865372 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:55.866035 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:55.866052 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:55.866900 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:55.866904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:55.867951 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:55.867962 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:55.868509 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:55.868514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:55.868694 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:55.869458 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:55.869477 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:55.871372 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:55.873313 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:55.873319 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:55.873334 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:55.873616 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:55.873936 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:55.873939 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:55.873941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:55.874085 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:55.874551 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:55.874559 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:55.874805 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 02:46:55.874993 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 02:46:55.875343 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":109,"threads_started":1}
I20260811 02:46:55.877494 27094 heartbeater.cc:347] Connected to a master server at 127.21.236.190:38227
I20260811 02:46:55.877775 27094 heartbeater.cc:464] Registering TS with master...
I20260811 02:46:55.878391 27094 heartbeater.cc:511] Master 127.21.236.190:38227 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
I20260811 02:46:55.879683 26533 ts_manager.cc:194] Registered new tserver with Master: 3ed67c4e1025471cb8a614960e493274 (127.21.236.130:38175)
I20260811 02:46:55.880311 26533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:50275
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:46:55.896261 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:38227
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:44693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:56.016510 27103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:56.016793 27103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:56.016839 27103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:56.020830 27103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:56.020892 27103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:56.020951 27103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:46:56.026229 27103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:44693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:38227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:56.027271 27103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:56.028295 27103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:56.038033 27113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:56.038075 27114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:56.038196 27120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:56.038089 27103 server_base.cc:1034] running on GCE node
I20260811 02:46:56.038769 27103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:56.039567 27103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:56.040764 27103 hybrid_clock.cc:648] HybridClock initialized: now 1786416416040741 us; error 36 us; skew 500 ppm
Aug 11 02:46:56 dist-test-slave-n326 krb5kdc[26441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416416, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:56.043735 27103 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:46:56.044556 27103 webserver.cc:460] Webserver started at http://127.21.236.131:41403/ using document root <none> and password file <none>
I20260811 02:46:56.045077 27103 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:56.045137 27103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:56.045303 27103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:56.047374 27103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "f79afdfd9054430f80d3d66e3c0803eb"
format_stamp: "Formatted at 2026-08-11 02:46:56 on dist-test-slave-n326"
server_key: "6779333a1a8a823f0383c32da075166d"
server_key_iv: "91091499120eaeb49634c79fa47681fa"
server_key_version: "encryptionkey@0"
I20260811 02:46:56.047890 27103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "f79afdfd9054430f80d3d66e3c0803eb"
format_stamp: "Formatted at 2026-08-11 02:46:56 on dist-test-slave-n326"
server_key: "6779333a1a8a823f0383c32da075166d"
server_key_iv: "91091499120eaeb49634c79fa47681fa"
server_key_version: "encryptionkey@0"
I20260811 02:46:56.051575 27103 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:46:56.054172 27129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:56.055624 27103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:46:56.055748 27103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "f79afdfd9054430f80d3d66e3c0803eb"
format_stamp: "Formatted at 2026-08-11 02:46:56 on dist-test-slave-n326"
server_key: "6779333a1a8a823f0383c32da075166d"
server_key_iv: "91091499120eaeb49634c79fa47681fa"
server_key_version: "encryptionkey@0"
I20260811 02:46:56.055842 27103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:56.069144 27103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:56.072775 27103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:56.072979 27103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:56.073611 27103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:46:56.074581 27103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:46:56.074633 27103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:56.074707 27103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:46:56.074748 27103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:56.084766 27103 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:43793
I20260811 02:46:56.084782 27340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:43793 every 8 connection(s)
I20260811 02:46:56.085740 27103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:46:56.094197 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 27103
I20260811 02:46:56.094300 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:46:56.094599 22450 external_mini_cluster.cc:1428] Setting key 4d53191030a0a81529a9e9078a5f3c47
Aug 11 02:46:56 dist-test-slave-n326 krb5kdc[26441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416416, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:56.099849 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:56.087637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:57011 (local address 127.21.236.190:38227)
0811 02:46:56.087770 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:56.087774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:56.088564 (+ 790us) server_negotiation.cc:407] Connection header received
0811 02:46:56.089889 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:56.089893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:56.089954 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:56.090056 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:56.091295 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:56.091835 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:56.092992 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:56.093212 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:56.093324 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:56.096213 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:56.096217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:56.096220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:56.096250 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:56.097713 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:56.098258 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:56.098261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:56.098262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:56.098339 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:56.098719 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:56.098722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:56.098723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:56.098899 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:56.099020 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:56.099627 (+ 607us) server_negotiation.cc:299] Negotiation successful
0811 02:46:56.099733 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:46:56.100838 27348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:56.087977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38227 (local address 127.21.236.131:57011)
0811 02:46:56.088415 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 02:46:56.088458 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:46:56.089677 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:46:56.090240 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:46:56.090247 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:46:56.090609 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:46:56.091143 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:56.091161 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:56.091962 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:56.091965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:56.092856 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:46:56.092865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:46:56.093366 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:46:56.093371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:46:56.093549 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:56.094243 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:46:56.094287 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:46:56.096043 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:46:56.097838 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:56.097849 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:56.097862 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:56.098157 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:56.098472 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:46:56.098476 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:46:56.098478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:46:56.098632 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:46:56.099021 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:46:56.099031 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:46:56.099343 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 02:46:56.099601 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 02:46:56.099886 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":121,"threads_started":1}
I20260811 02:46:56.101979 27343 heartbeater.cc:347] Connected to a master server at 127.21.236.190:38227
I20260811 02:46:56.102206 27343 heartbeater.cc:464] Registering TS with master...
I20260811 02:46:56.102658 27343 heartbeater.cc:511] Master 127.21.236.190:38227 requested a full tablet report, sending...
I20260811 02:46:56.103653 26533 ts_manager.cc:194] Registered new tserver with Master: f79afdfd9054430f80d3d66e3c0803eb (127.21.236.131:43793)
I20260811 02:46:56.104185 26533 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:57011
I20260811 02:46:56.108429 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 02:46:56.123293 26595 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:46:56.121617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48286 (local address 127.21.236.190:38227)
0811 02:46:56.121810 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:56.121815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:56.121989 (+ 174us) server_negotiation.cc:407] Connection header received
0811 02:46:56.122173 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:56.122176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:56.122219 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:56.122303 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:56.123139 (+ 836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48286: BlockingRecv error: recv got EOF from 127.0.0.1:48286 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 02:46:56.124577 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 26636
I20260811 02:46:56.139320 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 26861
I20260811 02:46:56.150558 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 27103
I20260811 02:46:56.165282 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 26472
2026-08-11T02:46:56Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2481 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:46:56 dist-test-slave-n326 krb5kdc[27367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:46:56 dist-test-slave-n326 krb5kdc[27367](info): set up 2 sockets
Aug 11 02:46:56 dist-test-slave-n326 krb5kdc[27367](info): commencing operation
krb5kdc: starting...
W20260811 02:46:57.542166 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.348s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:46:57 dist-test-slave-n326 krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:46:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:46:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:46:57.682475 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:33093
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:33093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:57.801772 27390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:57.802068 27390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:57.802140 27390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:57.806095 27390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:46:57.806185 27390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:57.806223 27390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:57.806277 27390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:46:57.806317 27390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:46:57.811424 27390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:33093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:33093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:57.812445 27390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:57.813446 27390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:57.822436 27406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:57.822383 27403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:57.822388 27401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:57.822803 27390 server_base.cc:1034] running on GCE node
I20260811 02:46:57.823482 27390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:57.824654 27390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:57.825868 27390 hybrid_clock.cc:648] HybridClock initialized: now 1786416417825836 us; error 43 us; skew 500 ppm
Aug 11 02:46:57 dist-test-slave-n326 krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416417, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:57.834156 27390 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:46:57.835178 27390 webserver.cc:460] Webserver started at http://127.21.236.190:33639/ using document root <none> and password file <none>
I20260811 02:46:57.835738 27390 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:57.835801 27390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:57.835981 27390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:57.838043 27390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "89df3293eefc47e1abf5369f49437f83"
format_stamp: "Formatted at 2026-08-11 02:46:57 on dist-test-slave-n326"
server_key: "67c13810002752b6f83e7a4228151399"
server_key_iv: "5178016becbb90f3a1dfb59b38990051"
server_key_version: "encryptionkey@0"
I20260811 02:46:57.838557 27390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "89df3293eefc47e1abf5369f49437f83"
format_stamp: "Formatted at 2026-08-11 02:46:57 on dist-test-slave-n326"
server_key: "67c13810002752b6f83e7a4228151399"
server_key_iv: "5178016becbb90f3a1dfb59b38990051"
server_key_version: "encryptionkey@0"
I20260811 02:46:57.842387 27390 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:46:57.844956 27419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:57.846282 27390 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:46:57.846431 27390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "89df3293eefc47e1abf5369f49437f83"
format_stamp: "Formatted at 2026-08-11 02:46:57 on dist-test-slave-n326"
server_key: "67c13810002752b6f83e7a4228151399"
server_key_iv: "5178016becbb90f3a1dfb59b38990051"
server_key_version: "encryptionkey@0"
I20260811 02:46:57.846539 27390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:57.864727 27390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:57.868418 27390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:57.868643 27390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:57.876261 27390 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:33093
I20260811 02:46:57.876566 27507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:33093 every 8 connection(s)
I20260811 02:46:57.877310 27390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:46:57.880277 27508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:57.880492 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 27390
I20260811 02:46:57.880623 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:46:57.880931 22450 external_mini_cluster.cc:1428] Setting key 4deb123a2a0d789cd2145068023f39b3
I20260811 02:46:57.887264 27508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83: Bootstrap starting.
I20260811 02:46:57.890105 27508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83: Neither blocks nor log segments found. Creating new log.
Aug 11 02:46:57 dist-test-slave-n326 krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:57.891036 27508 log.cc:824] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:57.893206 27508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83: No bootstrap required, opened a new log
I20260811 02:46:57.896013 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:57.882239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45924 (local address 127.21.236.190:33093)
0811 02:46:57.882685 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:57.882701 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:57.882736 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:46:57.883480 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:57.883499 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:57.883835 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:57.884122 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:57.885110 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:57.886110 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:57.887321 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:57.887848 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:57.887970 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:57.890811 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:57.890816 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:57.890830 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:57.890858 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:57.892809 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:57.893391 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:57.893397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:57.893402 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:57.893517 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:57.893923 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:57.893927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:57.893928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:57.894400 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:57.894558 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:57.894877 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 02:46:57.895249 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":137,"threads_started":1}
I20260811 02:46:57.897403 27508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89df3293eefc47e1abf5369f49437f83" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33093 } }
I20260811 02:46:57.897696 27508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:57.897761 27508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89df3293eefc47e1abf5369f49437f83, State: Initialized, Role: FOLLOWER
I20260811 02:46:57.898229 27508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89df3293eefc47e1abf5369f49437f83" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33093 } }
I20260811 02:46:57.898413 27508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:46:57.898484 27508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:46:57.898587 27508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:57.899173 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:46:57.899884 27508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89df3293eefc47e1abf5369f49437f83" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33093 } }
I20260811 02:46:57.900257 27508 leader_election.cc:302] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89df3293eefc47e1abf5369f49437f83; no voters:
I20260811 02:46:57.900642 27508 leader_election.cc:288] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:46:57.900746 27513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:57.901044 27513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [term 1 LEADER]: Becoming Leader. State: Replica: 89df3293eefc47e1abf5369f49437f83, State: Running, Role: LEADER
I20260811 02:46:57.901397 27513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89df3293eefc47e1abf5369f49437f83" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33093 } }
I20260811 02:46:57.902036 27508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:46:57.902961 27514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89df3293eefc47e1abf5369f49437f83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89df3293eefc47e1abf5369f49437f83" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33093 } } }
I20260811 02:46:57.903155 27514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [sys.catalog]: This master's current role is: LEADER
I20260811 02:46:57.903496 27517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89df3293eefc47e1abf5369f49437f83. Latest consensus state: current_term: 1 leader_uuid: "89df3293eefc47e1abf5369f49437f83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89df3293eefc47e1abf5369f49437f83" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33093 } } }
I20260811 02:46:57.903664 27517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83 [sys.catalog]: This master's current role is: LEADER
I20260811 02:46:57.904472 27532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:46:57.907610 27532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:46:57.914247 27532 catalog_manager.cc:1347] Generated new cluster ID: bba7b056017a4a5794083c18850cd0c9
I20260811 02:46:57.914323 27532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:46:57.921937 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:57.916172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45936 (local address 127.21.236.190:33093)
0811 02:46:57.916292 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:57.916296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:57.916577 (+ 281us) server_negotiation.cc:407] Connection header received
0811 02:46:57.916708 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:57.916711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:57.916768 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:57.916863 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:57.917454 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:57.918254 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:57.919456 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:57.919843 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:57.919959 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:57.920861 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:57.920865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:57.920868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:46:57.921458 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:57.921494 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:46:57.921514 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:57.921606 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:57.921714 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 02:46:57.921777 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 02:46:57.938483 27532 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:46:57.939564 27532 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:46:57.949110 27532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89df3293eefc47e1abf5369f49437f83: Generated new TSK 0
I20260811 02:46:57.949839 27532 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:46:57.962109 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 27390
2026-08-11T02:46:57Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1801 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:46:58 dist-test-slave-n326 krb5kdc[27566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:46:58 dist-test-slave-n326 krb5kdc[27566](info): set up 2 sockets
Aug 11 02:46:58 dist-test-slave-n326 krb5kdc[27566](info): commencing operation
krb5kdc: starting...
W20260811 02:46:59.369364 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.374s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:46:59 dist-test-slave-n326 krb5kdc[27566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:46:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:46:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:46:59.510617 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:43103
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:40521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:43103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:59.625079 27587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:59.625401 27587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:59.625478 27587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:59.629527 27587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:46:59.629628 27587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:59.629669 27587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:59.629719 27587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:46:59.629770 27587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:46:59.635053 27587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:40521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:43103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:43103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:59.636193 27587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:59.637220 27587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:59.646528 27597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:59.646546 27598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:59.646545 27603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:59.646945 27587 server_base.cc:1034] running on GCE node
I20260811 02:46:59.647636 27587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:59.649483 27587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:59.650689 27587 hybrid_clock.cc:648] HybridClock initialized: now 1786416419650672 us; error 37 us; skew 500 ppm
Aug 11 02:46:59 dist-test-slave-n326 krb5kdc[27566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416419, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:59.654079 27587 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:46:59.655205 27587 webserver.cc:460] Webserver started at http://127.21.236.190:46383/ using document root <none> and password file <none>
I20260811 02:46:59.655849 27587 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:59.655956 27587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:59.656220 27587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:59.658563 27587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "2d5e26428a614583bdfb6188b68190db"
format_stamp: "Formatted at 2026-08-11 02:46:59 on dist-test-slave-n326"
server_key: "aad45fde61c1db2029d937fcb8395193"
server_key_iv: "739e72bbb407b9ee39419c408cd12b98"
server_key_version: "encryptionkey@0"
I20260811 02:46:59.659281 27587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "2d5e26428a614583bdfb6188b68190db"
format_stamp: "Formatted at 2026-08-11 02:46:59 on dist-test-slave-n326"
server_key: "aad45fde61c1db2029d937fcb8395193"
server_key_iv: "739e72bbb407b9ee39419c408cd12b98"
server_key_version: "encryptionkey@0"
I20260811 02:46:59.663725 27587 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:46:59.666621 27612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:59.668109 27587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:46:59.668282 27587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "2d5e26428a614583bdfb6188b68190db"
format_stamp: "Formatted at 2026-08-11 02:46:59 on dist-test-slave-n326"
server_key: "aad45fde61c1db2029d937fcb8395193"
server_key_iv: "739e72bbb407b9ee39419c408cd12b98"
server_key_version: "encryptionkey@0"
I20260811 02:46:59.668416 27587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:46:59.702199 27587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:46:59.705945 27587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:46:59.706202 27587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:46:59.715152 27587 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:43103
I20260811 02:46:59.715476 27710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:43103 every 8 connection(s)
I20260811 02:46:59.716346 27587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:46:59.718248 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 27587
I20260811 02:46:59.718443 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:46:59.718768 22450 external_mini_cluster.cc:1428] Setting key 80fe75f44bebf10a03f31dd692137bb9
I20260811 02:46:59.719705 27712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:46:59.726548 27712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db: Bootstrap starting.
Aug 11 02:46:59 dist-test-slave-n326 krb5kdc[27566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:46:59.729825 27712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db: Neither blocks nor log segments found. Creating new log.
I20260811 02:46:59.730885 27712 log.cc:824] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db: Log is configured to *not* fsync() on all Append() calls
I20260811 02:46:59.733433 27712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db: No bootstrap required, opened a new log
I20260811 02:46:59.735538 27718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:46:59.720372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51808 (local address 127.21.236.190:43103)
0811 02:46:59.720844 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 02:46:59.720880 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 02:46:59.720920 (+ 40us) server_negotiation.cc:407] Connection header received
0811 02:46:59.721641 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:46:59.721659 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:46:59.722022 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:46:59.722324 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:46:59.723355 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:59.724492 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:59.725763 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:46:59.726328 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:46:59.726509 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:46:59.729679 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:46:59.729689 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:46:59.729709 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:46:59.729751 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:46:59.732286 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:59.732860 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:59.732866 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:59.732871 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:59.733000 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:46:59.733322 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:46:59.733324 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:46:59.733325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:46:59.733849 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:46:59.734020 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 02:46:59.734322 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 02:46:59.734666 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":152,"threads_started":1}
I20260811 02:46:59.737535 27712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5e26428a614583bdfb6188b68190db" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43103 } }
I20260811 02:46:59.737860 27712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:46:59.737968 27712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d5e26428a614583bdfb6188b68190db, State: Initialized, Role: FOLLOWER
I20260811 02:46:59.738423 27712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5e26428a614583bdfb6188b68190db" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43103 } }
I20260811 02:46:59.738586 27712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:46:59.738675 27712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:46:59.738799 27712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:46:59.739851 27712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5e26428a614583bdfb6188b68190db" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43103 } }
I20260811 02:46:59.740221 27712 leader_election.cc:302] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d5e26428a614583bdfb6188b68190db; no voters:
I20260811 02:46:59.740556 27712 leader_election.cc:288] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:46:59.740741 27721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:46:59.741043 27721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [term 1 LEADER]: Becoming Leader. State: Replica: 2d5e26428a614583bdfb6188b68190db, State: Running, Role: LEADER
I20260811 02:46:59.741425 27721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5e26428a614583bdfb6188b68190db" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43103 } }
I20260811 02:46:59.741922 27712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:46:59.743371 27724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d5e26428a614583bdfb6188b68190db. Latest consensus state: current_term: 1 leader_uuid: "2d5e26428a614583bdfb6188b68190db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5e26428a614583bdfb6188b68190db" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43103 } } }
I20260811 02:46:59.743528 27724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [sys.catalog]: This master's current role is: LEADER
I20260811 02:46:59.744172 27722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d5e26428a614583bdfb6188b68190db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d5e26428a614583bdfb6188b68190db" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43103 } } }
I20260811 02:46:59.744330 27722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db [sys.catalog]: This master's current role is: LEADER
W20260811 02:46:59.747215 27749 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:46:59.747304 27749 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:46:59.747383 27737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:46:59.748180 27737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:46:59.754159 27737 catalog_manager.cc:1347] Generated new cluster ID: eac8aa99fc904762b187d70f1d7dd2bc
I20260811 02:46:59.754237 27737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:46:59.766970 27737 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:46:59.767978 27737 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:46:59.777426 27737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d5e26428a614583bdfb6188b68190db: Generated new TSK 0
I20260811 02:46:59.778237 27737 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:46:59.814013 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43103
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:40521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:46:59.945356 27764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:46:59.945633 27764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:46:59.945677 27764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:46:59.949752 27764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:46:59.949817 27764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:46:59.949877 27764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:46:59.955166 27764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:40521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:46:59.956198 27764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:46:59.957170 27764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:46:59.967734 27779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:59.967747 27776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:46:59.967816 27775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:46:59.967959 27764 server_base.cc:1034] running on GCE node
I20260811 02:46:59.968509 27764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:46:59.969249 27764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:46:59.970468 27764 hybrid_clock.cc:648] HybridClock initialized: now 1786416419970452 us; error 28 us; skew 500 ppm
Aug 11 02:46:59 dist-test-slave-n326 krb5kdc[27566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416419, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:46:59.973814 27764 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:46:59.974722 27764 webserver.cc:460] Webserver started at http://127.21.236.129:43087/ using document root <none> and password file <none>
I20260811 02:46:59.975374 27764 fs_manager.cc:359] Metadata directory not provided
I20260811 02:46:59.975443 27764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:46:59.975659 27764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:46:59.977749 27764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "b2d2cca669a34b57ab485456cfd90c15"
format_stamp: "Formatted at 2026-08-11 02:46:59 on dist-test-slave-n326"
server_key: "c992360cb00c656e152f5a603a6667d9"
server_key_iv: "1da665db83adbaa348903f39705d40c2"
server_key_version: "encryptionkey@0"
I20260811 02:46:59.978346 27764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "b2d2cca669a34b57ab485456cfd90c15"
format_stamp: "Formatted at 2026-08-11 02:46:59 on dist-test-slave-n326"
server_key: "c992360cb00c656e152f5a603a6667d9"
server_key_iv: "1da665db83adbaa348903f39705d40c2"
server_key_version: "encryptionkey@0"
I20260811 02:46:59.982466 27764 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 02:46:59.985139 27789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:46:59.986567 27764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:46:59.986733 27764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "b2d2cca669a34b57ab485456cfd90c15"
format_stamp: "Formatted at 2026-08-11 02:46:59 on dist-test-slave-n326"
server_key: "c992360cb00c656e152f5a603a6667d9"
server_key_iv: "1da665db83adbaa348903f39705d40c2"
server_key_version: "encryptionkey@0"
I20260811 02:46:59.986863 27764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:00.014829 27764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:00.018778 27764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:00.019001 27764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:00.019758 27764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:00.020788 27764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:00.020843 27764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:00.020920 27764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:00.020955 27764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:00.030683 27764 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:42193
I20260811 02:47:00.030714 27982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:42193 every 8 connection(s)
I20260811 02:47:00.031718 27764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:47:00.033110 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 27764
I20260811 02:47:00.033262 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:47:00.033638 22450 external_mini_cluster.cc:1428] Setting key e3b81c269a264f443f05704a104c4df3
Aug 11 02:47:00 dist-test-slave-n326 krb5kdc[27566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416419, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
I20260811 02:47:00.048640 27718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:00.033789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:47437 (local address 127.21.236.190:43103)
0811 02:47:00.033938 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:00.033944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:00.035111 (+ 1167us) server_negotiation.cc:407] Connection header received
0811 02:47:00.036751 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:00.036758 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:00.036825 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:00.036923 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:00.038240 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:00.039132 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:00.040389 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:00.040793 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:00.040931 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:00.044097 (+ 3166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:00.044100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:00.044102 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:00.044130 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:00.045976 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:00.046625 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:00.046628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:00.046629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:00.046715 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:00.047216 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:00.047219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:00.047221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:00.047464 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:00.047555 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:00.048242 (+ 687us) server_negotiation.cc:299] Negotiation successful
0811 02:47:00.048402 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 02:47:00.049502 27986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:00.034328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43103 (local address 127.21.236.129:47437)
0811 02:47:00.034906 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:00.034965 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:00.036511 (+ 1546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:00.037115 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:00.037123 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:00.037526 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:00.038060 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:00.038076 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:00.039277 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:00.039280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:00.040258 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:00.040269 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:00.040961 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:00.040964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:00.041134 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:00.041825 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:00.041866 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:00.043888 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:00.046124 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:00.046129 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:00.046143 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:00.046473 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:00.046847 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:00.046850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:00.046852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:00.047053 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:00.047579 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:00.047585 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:00.047945 (+ 360us) client_negotiation.cc:769] Sending connection context
0811 02:47:00.048173 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 02:47:00.048538 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":165,"threads_started":1}
I20260811 02:47:00.050865 27983 heartbeater.cc:347] Connected to a master server at 127.21.236.190:43103
I20260811 02:47:00.051214 27983 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:00.051854 27983 heartbeater.cc:511] Master 127.21.236.190:43103 requested a full tablet report, sending...
I20260811 02:47:00.053584 27643 ts_manager.cc:194] Registered new tserver with Master: b2d2cca669a34b57ab485456cfd90c15 (127.21.236.129:42193)
I20260811 02:47:00.054754 27643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:47437
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:47:00.066218 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43103
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:40521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:00.185020 27992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:00.185343 27992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:00.185433 27992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:00.189481 27992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:00.189586 27992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:00.189682 27992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:47:00.195312 27992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:40521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:00.196461 27992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:00.197566 27992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:00.208961 28004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:00.209136 28000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:00.208988 28001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:00.209496 27992 server_base.cc:1034] running on GCE node
I20260811 02:47:00.209944 27992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:00.210620 27992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:00.211845 27992 hybrid_clock.cc:648] HybridClock initialized: now 1786416420211820 us; error 45 us; skew 500 ppm
Aug 11 02:47:00 dist-test-slave-n326 krb5kdc[27566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416420, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:00.215167 27992 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:47:00.216063 27992 webserver.cc:460] Webserver started at http://127.21.236.130:38975/ using document root <none> and password file <none>
I20260811 02:47:00.216684 27992 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:00.216749 27992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:00.216980 27992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:00.219218 27992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "1c3822e658e44bc7847f53b5458efdd7"
format_stamp: "Formatted at 2026-08-11 02:47:00 on dist-test-slave-n326"
server_key: "69d2d2f2785d491fab39efbd1af5e95c"
server_key_iv: "bbf6634154e6b1e00f159a01bfe27322"
server_key_version: "encryptionkey@0"
I20260811 02:47:00.219815 27992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "1c3822e658e44bc7847f53b5458efdd7"
format_stamp: "Formatted at 2026-08-11 02:47:00 on dist-test-slave-n326"
server_key: "69d2d2f2785d491fab39efbd1af5e95c"
server_key_iv: "bbf6634154e6b1e00f159a01bfe27322"
server_key_version: "encryptionkey@0"
I20260811 02:47:00.223980 27992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:47:00.226841 28016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:00.228392 27992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:47:00.228561 27992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "1c3822e658e44bc7847f53b5458efdd7"
format_stamp: "Formatted at 2026-08-11 02:47:00 on dist-test-slave-n326"
server_key: "69d2d2f2785d491fab39efbd1af5e95c"
server_key_iv: "bbf6634154e6b1e00f159a01bfe27322"
server_key_version: "encryptionkey@0"
I20260811 02:47:00.228686 27992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:00.247407 27992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:00.251222 27992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:00.251458 27992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:00.252133 27992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:00.253057 27992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:00.253110 27992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:00.253186 27992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:00.253228 27992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:00.263402 27992 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:32883
I20260811 02:47:00.263427 28234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:32883 every 8 connection(s)
I20260811 02:47:00.264446 27992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:47:00.274333 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 27992
I20260811 02:47:00.274501 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:47:00.274895 22450 external_mini_cluster.cc:1428] Setting key 43f8f8d8527763358113c59730dfc376
Aug 11 02:47:00 dist-test-slave-n326 krb5kdc[27566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416420, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:00.280783 27718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:00.266581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:60099 (local address 127.21.236.190:43103)
0811 02:47:00.266749 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:00.266752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:00.267739 (+ 987us) server_negotiation.cc:407] Connection header received
0811 02:47:00.269197 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:00.269201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:00.269256 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:00.269339 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:00.270608 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:00.271222 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:00.272660 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:00.272951 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:00.273046 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:00.276140 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:00.276143 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:00.276145 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:00.276170 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:00.278280 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:00.278901 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:00.278908 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:00.278912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:00.279027 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:00.279466 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:00.279471 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:00.279473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:00.279753 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:00.279893 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:00.280486 (+ 593us) server_negotiation.cc:299] Negotiation successful
0811 02:47:00.280652 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 02:47:00.281945 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:00.267005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43103 (local address 127.21.236.130:60099)
0811 02:47:00.267572 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:00.267619 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:00.268980 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:00.269547 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:00.269554 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:00.269939 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:00.270446 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:00.270461 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:00.271401 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:00.271404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:00.272469 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:00.272481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:00.273115 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:00.273118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:00.273274 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:00.273998 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:00.274014 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:00.275973 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:00.278458 (+ 2485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:00.278463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:00.278476 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:00.278760 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:00.279203 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:00.279205 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:00.279207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:00.279355 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:00.279917 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:00.279923 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:00.280178 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 02:47:00.280352 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 02:47:00.280742 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":123,"threads_started":1}
I20260811 02:47:00.283337 28235 heartbeater.cc:347] Connected to a master server at 127.21.236.190:43103
I20260811 02:47:00.283608 28235 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:00.284137 28235 heartbeater.cc:511] Master 127.21.236.190:43103 requested a full tablet report, sending...
I20260811 02:47:00.285267 27643 ts_manager.cc:194] Registered new tserver with Master: 1c3822e658e44bc7847f53b5458efdd7 (127.21.236.130:32883)
I20260811 02:47:00.285857 27643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:60099
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:47:00.306870 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43103
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:40521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:00.420789 28247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:00.421087 28247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:00.421132 28247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:00.425143 28247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:00.425210 28247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:00.425271 28247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:47:00.430599 28247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:40521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:00.431738 28247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:00.432880 28247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:00.443003 28260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:00.443077 28257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:00.443048 28247 server_base.cc:1034] running on GCE node
W20260811 02:47:00.443603 28258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:00.444110 28247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:00.444782 28247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:00.445960 28247 hybrid_clock.cc:648] HybridClock initialized: now 1786416420445941 us; error 34 us; skew 500 ppm
Aug 11 02:47:00 dist-test-slave-n326 krb5kdc[27566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416420, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:00.449199 28247 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:47:00.450194 28247 webserver.cc:460] Webserver started at http://127.21.236.131:34243/ using document root <none> and password file <none>
I20260811 02:47:00.450800 28247 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:00.450897 28247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:00.451236 28247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:00.453419 28247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "a2a5dbd28cf6457da170a9da614af077"
format_stamp: "Formatted at 2026-08-11 02:47:00 on dist-test-slave-n326"
server_key: "a583188cbf098469b057edc846c67b68"
server_key_iv: "3d165f875d04cd06c7080f736f4eb1b2"
server_key_version: "encryptionkey@0"
I20260811 02:47:00.454015 28247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "a2a5dbd28cf6457da170a9da614af077"
format_stamp: "Formatted at 2026-08-11 02:47:00 on dist-test-slave-n326"
server_key: "a583188cbf098469b057edc846c67b68"
server_key_iv: "3d165f875d04cd06c7080f736f4eb1b2"
server_key_version: "encryptionkey@0"
I20260811 02:47:00.457994 28247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:47:00.460853 28273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:00.462275 28247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:47:00.462441 28247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "a2a5dbd28cf6457da170a9da614af077"
format_stamp: "Formatted at 2026-08-11 02:47:00 on dist-test-slave-n326"
server_key: "a583188cbf098469b057edc846c67b68"
server_key_iv: "3d165f875d04cd06c7080f736f4eb1b2"
server_key_version: "encryptionkey@0"
I20260811 02:47:00.462585 28247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:00.473639 28247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:00.477089 28247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:00.477341 28247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:00.478029 28247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:00.478997 28247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:00.479126 28247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:00.479231 28247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:00.479288 28247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:00.489470 28247 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:35483
I20260811 02:47:00.489490 28459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:35483 every 8 connection(s)
I20260811 02:47:00.490545 28247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:47:00.494354 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 28247
I20260811 02:47:00.494517 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:47:00.494843 22450 external_mini_cluster.cc:1428] Setting key 8fa932a69523ae439a7dc7e26cec5142
Aug 11 02:47:00 dist-test-slave-n326 krb5kdc[27566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416420, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:00.505522 27718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:00.492716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:53921 (local address 127.21.236.190:43103)
0811 02:47:00.492883 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:00.492889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:00.493906 (+ 1017us) server_negotiation.cc:407] Connection header received
0811 02:47:00.495602 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:00.495606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:00.495731 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:00.495857 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:00.497099 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:00.497648 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:00.498797 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:00.499029 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:00.499211 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:00.501839 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:00.501842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:00.501844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:00.501870 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:00.503284 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:00.503881 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:00.503887 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:00.503891 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:00.503978 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:00.504350 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:00.504354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:00.504355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:00.504562 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:00.504680 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:00.505200 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 02:47:00.505334 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:47:00.506361 28469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:00.493229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43103 (local address 127.21.236.131:53921)
0811 02:47:00.493729 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:00.493776 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:00.495252 (+ 1476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:00.496059 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:00.496066 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:00.496450 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:00.496947 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:00.496961 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:00.497778 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:00.497782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:00.498664 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:00.498673 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:00.499248 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:00.499252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:00.499439 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:00.499998 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:00.500015 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:00.501653 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:00.503471 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:00.503478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:00.503494 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:00.503772 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:00.504086 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:00.504089 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:00.504091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:00.504238 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:00.504676 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:00.504683 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:00.504953 (+ 270us) client_negotiation.cc:769] Sending connection context
0811 02:47:00.505158 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 02:47:00.505500 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":124,"threads_started":1}
I20260811 02:47:00.507725 28461 heartbeater.cc:347] Connected to a master server at 127.21.236.190:43103
I20260811 02:47:00.508040 28461 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:00.508687 28461 heartbeater.cc:511] Master 127.21.236.190:43103 requested a full tablet report, sending...
I20260811 02:47:00.509853 27643 ts_manager.cc:194] Registered new tserver with Master: a2a5dbd28cf6457da170a9da614af077 (127.21.236.131:35483)
I20260811 02:47:00.510464 27643 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:53921
I20260811 02:47:00.521781 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 02:47:00 dist-test-slave-n326 krb5kdc[27566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416420, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 02:47:00 dist-test-slave-n326 krb5kdc[27566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416420, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:00.552907 27718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:00.543655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51810 (local address 127.21.236.190:43103)
0811 02:47:00.543846 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:00.543851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:00.544014 (+ 163us) server_negotiation.cc:407] Connection header received
0811 02:47:00.544157 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:00.544160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:00.544211 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:00.544293 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:00.545084 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:00.545634 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:00.546603 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:00.546820 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:00.546903 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:00.549527 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:00.549531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:00.549534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:00.549577 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:00.551049 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:00.551575 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:00.551579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:00.551581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:00.551668 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:00.551962 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:00.551965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:00.551967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:00.552202 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:00.552344 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:00.552599 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 02:47:00.552734 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 02:47:00.553339 27643 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51810
I20260811 02:47:00.554309 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 27764
I20260811 02:47:00.585204 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 27992
I20260811 02:47:00.603194 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 28247
I20260811 02:47:00.617241 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 27587
2026-08-11T02:47:00Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2650 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:47:00 dist-test-slave-n326 krb5kdc[28489](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:47:00 dist-test-slave-n326 krb5kdc[28489](info): set up 2 sockets
Aug 11 02:47:00 dist-test-slave-n326 krb5kdc[28489](info): commencing operation
krb5kdc: starting...
W20260811 02:47:02.031440 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.386s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:47:02 dist-test-slave-n326 krb5kdc[28489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:47:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:47:02.174822 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:39987
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:40059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:39987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:02.303589 28511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:02.303908 28511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:02.304212 28511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:02.309031 28511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:02.309114 28511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:02.309135 28511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:02.309154 28511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:02.309170 28511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:47:02.314828 28511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:40059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:39987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:39987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:02.316071 28511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:02.317202 28511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:02.326225 28527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:02.326287 28523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:02.326298 28524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:02.326629 28511 server_base.cc:1034] running on GCE node
I20260811 02:47:02.327355 28511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:02.328526 28511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:02.329730 28511 hybrid_clock.cc:648] HybridClock initialized: now 1786416422329695 us; error 47 us; skew 500 ppm
Aug 11 02:47:02 dist-test-slave-n326 krb5kdc[28489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416422, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:02.332973 28511 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:47:02.334100 28511 webserver.cc:460] Webserver started at http://127.21.236.190:33561/ using document root <none> and password file <none>
I20260811 02:47:02.334748 28511 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:02.334849 28511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:02.335196 28511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:02.337585 28511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "a1b7ca0c1d944c8c91532c0cdfeb2f6b"
format_stamp: "Formatted at 2026-08-11 02:47:02 on dist-test-slave-n326"
server_key: "104ce0b540ed0cf0b891c4525a1874d0"
server_key_iv: "0ec8bb32fcea5072bae9ce577940a652"
server_key_version: "encryptionkey@0"
I20260811 02:47:02.338217 28511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "a1b7ca0c1d944c8c91532c0cdfeb2f6b"
format_stamp: "Formatted at 2026-08-11 02:47:02 on dist-test-slave-n326"
server_key: "104ce0b540ed0cf0b891c4525a1874d0"
server_key_iv: "0ec8bb32fcea5072bae9ce577940a652"
server_key_version: "encryptionkey@0"
I20260811 02:47:02.342389 28511 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:47:02.345163 28536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:02.346633 28511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:47:02.346791 28511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "a1b7ca0c1d944c8c91532c0cdfeb2f6b"
format_stamp: "Formatted at 2026-08-11 02:47:02 on dist-test-slave-n326"
server_key: "104ce0b540ed0cf0b891c4525a1874d0"
server_key_iv: "0ec8bb32fcea5072bae9ce577940a652"
server_key_version: "encryptionkey@0"
I20260811 02:47:02.346917 28511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:02.370118 28511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:02.373361 28511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:02.373595 28511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:02.381563 28511 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:39987
I20260811 02:47:02.381585 28644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:39987 every 8 connection(s)
I20260811 02:47:02.382578 28511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:02.383461 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 28511
I20260811 02:47:02.383626 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:47:02.383991 22450 external_mini_cluster.cc:1428] Setting key 3a66ca9f6ac726da92bbee7870325efa
I20260811 02:47:02.386549 28646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:02.394150 28646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b: Bootstrap starting.
Aug 11 02:47:02 dist-test-slave-n326 krb5kdc[28489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:02.397244 28646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:02.398249 28646 log.cc:824] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:02.400590 28646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b: No bootstrap required, opened a new log
I20260811 02:47:02.401345 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:02.385713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.21.236.190:39987)
0811 02:47:02.386312 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:02.386329 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:02.386377 (+ 48us) server_negotiation.cc:407] Connection header received
0811 02:47:02.387266 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:02.387288 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:02.387661 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:02.387974 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:02.389041 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:02.390152 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:02.391495 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:02.392022 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:02.392168 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:02.395540 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:02.395548 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:02.395569 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:02.395614 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:02.397890 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:02.398470 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:02.398478 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:02.398486 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:02.398639 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:02.399024 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:02.399027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:02.399029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:02.399552 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:02.399777 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:02.400180 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 02:47:02.400518 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":182,"threads_started":1}
I20260811 02:47:02.405224 28646 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b7ca0c1d944c8c91532c0cdfeb2f6b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39987 } }
I20260811 02:47:02.405500 28646 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:02.405588 28646 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b7ca0c1d944c8c91532c0cdfeb2f6b, State: Initialized, Role: FOLLOWER
I20260811 02:47:02.406107 28646 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b7ca0c1d944c8c91532c0cdfeb2f6b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39987 } }
I20260811 02:47:02.406273 28646 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:02.406365 28646 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:02.406488 28646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:02.407655 28646 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b7ca0c1d944c8c91532c0cdfeb2f6b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39987 } }
I20260811 02:47:02.408033 28646 leader_election.cc:302] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1b7ca0c1d944c8c91532c0cdfeb2f6b; no voters:
I20260811 02:47:02.408411 28646 leader_election.cc:288] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:47:02.408581 28657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:02.408850 28657 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [term 1 LEADER]: Becoming Leader. State: Replica: a1b7ca0c1d944c8c91532c0cdfeb2f6b, State: Running, Role: LEADER
I20260811 02:47:02.409273 28657 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b7ca0c1d944c8c91532c0cdfeb2f6b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39987 } }
I20260811 02:47:02.409806 28646 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:02.411023 28661 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1b7ca0c1d944c8c91532c0cdfeb2f6b. Latest consensus state: current_term: 1 leader_uuid: "a1b7ca0c1d944c8c91532c0cdfeb2f6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b7ca0c1d944c8c91532c0cdfeb2f6b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39987 } } }
I20260811 02:47:02.411192 28661 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:02.411506 28659 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1b7ca0c1d944c8c91532c0cdfeb2f6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1b7ca0c1d944c8c91532c0cdfeb2f6b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39987 } } }
I20260811 02:47:02.411608 28659 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:02.411613 28673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:02.414659 28673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:02.421502 28673 catalog_manager.cc:1347] Generated new cluster ID: f910f901a7274cef94b744d33df41641
I20260811 02:47:02.421622 28673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:02.432081 28673 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:47:02.433499 28673 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:02.444751 28673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1b7ca0c1d944c8c91532c0cdfeb2f6b: Generated new TSK 0
I20260811 02:47:02.445515 28673 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:47:02.482384 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39987
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:40059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:02.601246 28691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:02.601554 28691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:02.601599 28691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:02.605810 28691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:02.605883 28691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:02.605943 28691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:47:02.611754 28691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:40059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:02.612869 28691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:02.613930 28691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:02.624029 28701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:02.623883 28702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:02.623931 28705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:02.624408 28691 server_base.cc:1034] running on GCE node
I20260811 02:47:02.624874 28691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:02.625563 28691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:02.626767 28691 hybrid_clock.cc:648] HybridClock initialized: now 1786416422626750 us; error 30 us; skew 500 ppm
Aug 11 02:47:02 dist-test-slave-n326 krb5kdc[28489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416422, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:02.629784 28691 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:47:02.630740 28691 webserver.cc:460] Webserver started at http://127.21.236.129:44913/ using document root <none> and password file <none>
I20260811 02:47:02.631368 28691 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:02.631433 28691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:02.631608 28691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:02.633701 28691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "e269898df59f41a8a26243a0c971a86a"
format_stamp: "Formatted at 2026-08-11 02:47:02 on dist-test-slave-n326"
server_key: "7b62856a50773e466784ba5ebec904c2"
server_key_iv: "6fe2efac275df619bde8c64769cdb355"
server_key_version: "encryptionkey@0"
I20260811 02:47:02.634250 28691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "e269898df59f41a8a26243a0c971a86a"
format_stamp: "Formatted at 2026-08-11 02:47:02 on dist-test-slave-n326"
server_key: "7b62856a50773e466784ba5ebec904c2"
server_key_iv: "6fe2efac275df619bde8c64769cdb355"
server_key_version: "encryptionkey@0"
I20260811 02:47:02.638250 28691 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:47:02.640894 28723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:02.642242 28691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 02:47:02.642376 28691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "e269898df59f41a8a26243a0c971a86a"
format_stamp: "Formatted at 2026-08-11 02:47:02 on dist-test-slave-n326"
server_key: "7b62856a50773e466784ba5ebec904c2"
server_key_iv: "6fe2efac275df619bde8c64769cdb355"
server_key_version: "encryptionkey@0"
I20260811 02:47:02.642500 28691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:02.655306 28691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:02.658524 28691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:02.658746 28691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:02.659474 28691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:02.660362 28691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:02.660419 28691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:02.660490 28691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:02.660521 28691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:02.669694 28691 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:36837
I20260811 02:47:02.669721 28915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:36837 every 8 connection(s)
I20260811 02:47:02.670606 28691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:47:02.680071 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 28691
I20260811 02:47:02.680187 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:47:02.680598 22450 external_mini_cluster.cc:1428] Setting key 5148af407a5d146c4dae907494e32ee8
Aug 11 02:47:02 dist-test-slave-n326 krb5kdc[28489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416422, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:02.686000 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:02.672596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:58195 (local address 127.21.236.190:39987)
0811 02:47:02.672749 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:02.672753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:02.673537 (+ 784us) server_negotiation.cc:407] Connection header received
0811 02:47:02.675014 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:02.675018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:02.675090 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:02.675170 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:02.676433 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:02.677009 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:02.678220 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:02.678472 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:02.678561 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:02.681558 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:02.681562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:02.681565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:02.681602 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:02.683289 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:02.684037 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:02.684040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:02.684042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:02.684131 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:02.684585 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:02.684592 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:02.684593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:02.684782 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:02.684930 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:02.685641 (+ 711us) server_negotiation.cc:299] Negotiation successful
0811 02:47:02.685805 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:47:02.686990 28919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:02.672864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:39987 (local address 127.21.236.129:58195)
0811 02:47:02.673333 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:02.673378 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:02.674783 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:02.675352 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:02.675359 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:02.675772 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:02.676274 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:02.676287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:02.677127 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:02.677130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:02.678094 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:02.678104 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:02.678621 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:02.678624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:02.678772 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:02.679480 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:02.679498 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:02.681377 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:02.683421 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:02.683432 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:02.683455 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:02.683907 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:02.684238 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:02.684245 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:02.684250 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:02.684473 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:02.684929 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:02.684937 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:02.685214 (+ 277us) client_negotiation.cc:769] Sending connection context
0811 02:47:02.685593 (+ 379us) client_negotiation.cc:240] Negotiation successful
0811 02:47:02.686015 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":118,"threads_started":1}
I20260811 02:47:02.688757 28916 heartbeater.cc:347] Connected to a master server at 127.21.236.190:39987
I20260811 02:47:02.689114 28916 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:02.689934 28916 heartbeater.cc:511] Master 127.21.236.190:39987 requested a full tablet report, sending...
I20260811 02:47:02.691898 28565 ts_manager.cc:194] Registered new tserver with Master: e269898df59f41a8a26243a0c971a86a (127.21.236.129:36837)
I20260811 02:47:02.693745 28565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:58195
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:47:02.714491 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39987
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:40059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:02.839682 28926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:02.840018 28926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:02.840070 28926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:02.844969 28926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:02.845044 28926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:02.845113 28926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:47:02.851632 28926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:40059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:02.852782 28926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:02.854002 28926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:02.864951 28933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:02.865020 28935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:02.865017 28931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:02.865367 28926 server_base.cc:1034] running on GCE node
I20260811 02:47:02.865777 28926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:02.866465 28926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:02.867666 28926 hybrid_clock.cc:648] HybridClock initialized: now 1786416422867649 us; error 28 us; skew 500 ppm
Aug 11 02:47:02 dist-test-slave-n326 krb5kdc[28489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416422, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:02.880283 28926 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:47:02.881417 28926 webserver.cc:460] Webserver started at http://127.21.236.130:41979/ using document root <none> and password file <none>
I20260811 02:47:02.882036 28926 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:02.882107 28926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:02.882302 28926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:02.885110 28926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "c86310fe3286495e964b192bd7dc3e28"
format_stamp: "Formatted at 2026-08-11 02:47:02 on dist-test-slave-n326"
server_key: "34e30b616c125238bfac72b9a67f6156"
server_key_iv: "8569ecd7647408a44cdf56c6308dddfe"
server_key_version: "encryptionkey@0"
I20260811 02:47:02.885804 28926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "c86310fe3286495e964b192bd7dc3e28"
format_stamp: "Formatted at 2026-08-11 02:47:02 on dist-test-slave-n326"
server_key: "34e30b616c125238bfac72b9a67f6156"
server_key_iv: "8569ecd7647408a44cdf56c6308dddfe"
server_key_version: "encryptionkey@0"
I20260811 02:47:02.890542 28926 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 02:47:02.893428 28945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:02.894825 28926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:47:02.894985 28926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "c86310fe3286495e964b192bd7dc3e28"
format_stamp: "Formatted at 2026-08-11 02:47:02 on dist-test-slave-n326"
server_key: "34e30b616c125238bfac72b9a67f6156"
server_key_iv: "8569ecd7647408a44cdf56c6308dddfe"
server_key_version: "encryptionkey@0"
I20260811 02:47:02.895185 28926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:02.919881 28926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:02.923691 28926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:02.923933 28926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:02.924638 28926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:02.925592 28926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:02.925645 28926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:02.925689 28926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:02.925755 28926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:02.936273 28926 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:43229
I20260811 02:47:02.936304 29128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:43229 every 8 connection(s)
I20260811 02:47:02.937317 28926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:47:02.943696 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 28926
I20260811 02:47:02.943823 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:47:02.944114 22450 external_mini_cluster.cc:1428] Setting key 1ec9214b46387812958658938c554b7c
Aug 11 02:47:02 dist-test-slave-n326 krb5kdc[28489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416422, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:02.953756 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:02.939434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:46721 (local address 127.21.236.190:39987)
0811 02:47:02.939611 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:02.939615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:02.940451 (+ 836us) server_negotiation.cc:407] Connection header received
0811 02:47:02.941983 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:02.941986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:02.942035 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:02.942116 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:02.943535 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:02.944357 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:02.946251 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:02.946704 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:02.946838 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:02.949562 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:02.949568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:02.949573 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:02.949610 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:02.951533 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:02.952189 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:02.952194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:02.952196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:02.952284 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:02.952632 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:02.952637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:02.952639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:02.952826 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:02.952939 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:02.953460 (+ 521us) server_negotiation.cc:299] Negotiation successful
0811 02:47:02.953585 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 02:47:02.954644 29131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:02.939823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:39987 (local address 127.21.236.130:46721)
0811 02:47:02.940293 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:02.940340 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:02.941752 (+ 1412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:02.942298 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:02.942305 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:02.942722 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:02.943356 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:02.943374 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:02.944555 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:02.944560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:02.946094 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:02.946105 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:02.946880 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:02.946883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:02.947141 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:02.947826 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:02.947843 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:02.949371 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:02.951654 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:02.951663 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:02.951682 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:02.952077 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:02.952381 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:02.952385 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:02.952387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:02.952532 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:02.952943 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:02.952952 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:02.953204 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 02:47:02.953400 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 02:47:02.953734 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
I20260811 02:47:02.956082 29129 heartbeater.cc:347] Connected to a master server at 127.21.236.190:39987
I20260811 02:47:02.956406 29129 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:02.956991 29129 heartbeater.cc:511] Master 127.21.236.190:39987 requested a full tablet report, sending...
I20260811 02:47:02.958268 28565 ts_manager.cc:194] Registered new tserver with Master: c86310fe3286495e964b192bd7dc3e28 (127.21.236.130:43229)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
I20260811 02:47:02.959174 28565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:46721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:47:02.976282 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39987
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:40059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:03.092360 29142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:03.092810 29142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:03.092913 29142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:03.097338 29142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:03.097405 29142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:03.097463 29142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:47:03.103047 29142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:40059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:03.104192 29142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:03.105235 29142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:03.115581 29152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:03.115590 29149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:03.115605 29147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:03.115803 29142 server_base.cc:1034] running on GCE node
I20260811 02:47:03.116528 29142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:03.117219 29142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:03.118412 29142 hybrid_clock.cc:648] HybridClock initialized: now 1786416423118395 us; error 40 us; skew 500 ppm
Aug 11 02:47:03 dist-test-slave-n326 krb5kdc[28489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416423, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:03.121803 29142 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:47:03.122895 29142 webserver.cc:460] Webserver started at http://127.21.236.131:37469/ using document root <none> and password file <none>
I20260811 02:47:03.123700 29142 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:03.123778 29142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:03.124027 29142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:03.126358 29142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "ac3d93cbe2e24a7fae1f548789c6db53"
format_stamp: "Formatted at 2026-08-11 02:47:03 on dist-test-slave-n326"
server_key: "977e8e7ca35be4d5c6e0c19320f3933e"
server_key_iv: "89cfd398cc157e893a6d164d6900d145"
server_key_version: "encryptionkey@0"
I20260811 02:47:03.126996 29142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "ac3d93cbe2e24a7fae1f548789c6db53"
format_stamp: "Formatted at 2026-08-11 02:47:03 on dist-test-slave-n326"
server_key: "977e8e7ca35be4d5c6e0c19320f3933e"
server_key_iv: "89cfd398cc157e893a6d164d6900d145"
server_key_version: "encryptionkey@0"
I20260811 02:47:03.131204 29142 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:47:03.134006 29164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:03.135478 29142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:47:03.135588 29142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "ac3d93cbe2e24a7fae1f548789c6db53"
format_stamp: "Formatted at 2026-08-11 02:47:03 on dist-test-slave-n326"
server_key: "977e8e7ca35be4d5c6e0c19320f3933e"
server_key_iv: "89cfd398cc157e893a6d164d6900d145"
server_key_version: "encryptionkey@0"
I20260811 02:47:03.135723 29142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:03.154325 29142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:03.157819 29142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:03.158022 29142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:03.158689 29142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:03.159633 29142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:03.159682 29142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:03.159744 29142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:03.159778 29142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:03.169353 29142 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:45519
I20260811 02:47:03.169379 29368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:45519 every 8 connection(s)
I20260811 02:47:03.170264 29142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:47:03.174226 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 29142
I20260811 02:47:03.174358 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:47:03.174717 22450 external_mini_cluster.cc:1428] Setting key bd54a4568971ceffeccaebb90ad9b914
Aug 11 02:47:03 dist-test-slave-n326 krb5kdc[28489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416423, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:03.186038 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:03.172204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:37565 (local address 127.21.236.190:39987)
0811 02:47:03.172341 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:03.172344 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:03.173142 (+ 798us) server_negotiation.cc:407] Connection header received
0811 02:47:03.174740 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:03.174743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:03.174822 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:03.174913 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:03.176558 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:03.177251 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:03.178401 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:03.178630 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:03.178714 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:03.181889 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:03.181893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:03.181895 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:03.181925 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:03.183608 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:03.184223 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:03.184227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:03.184229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:03.184326 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:03.184808 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:03.184811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:03.184813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:03.185035 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:03.185140 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:03.185767 (+ 627us) server_negotiation.cc:299] Negotiation successful
0811 02:47:03.185896 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:47:03.187003 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:03.172516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:39987 (local address 127.21.236.131:37565)
0811 02:47:03.172967 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:03.173021 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:03.174488 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:03.175160 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:03.175168 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:03.175691 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:03.176370 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:03.176386 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:03.177401 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:03.177403 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:03.178283 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:03.178293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:03.178931 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:03.178935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:03.179169 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:03.179955 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:03.179972 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:03.181715 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:03.183735 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:03.183741 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:03.183756 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:03.184079 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:03.184499 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:03.184502 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:03.184503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:03.184688 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:03.185164 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:03.185171 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:03.185478 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 02:47:03.185692 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 02:47:03.186022 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":113,"threads_started":1}
I20260811 02:47:03.188462 29369 heartbeater.cc:347] Connected to a master server at 127.21.236.190:39987
I20260811 02:47:03.188773 29369 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:03.189422 29369 heartbeater.cc:511] Master 127.21.236.190:39987 requested a full tablet report, sending...
I20260811 02:47:03.190604 28572 ts_manager.cc:194] Registered new tserver with Master: ac3d93cbe2e24a7fae1f548789c6db53 (127.21.236.131:45519)
I20260811 02:47:03.191321 28572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:37565
I20260811 02:47:03.201315 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 02:47:03 dist-test-slave-n326 krb5kdc[28489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:47:03 dist-test-slave-n326 krb5kdc[28489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.129@KRBTEST.COM
I20260811 02:47:03.231377 29384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:03.221350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48236 (local address 127.21.236.129:36837)
0811 02:47:03.221707 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:03.221721 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:03.221745 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:47:03.221800 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:03.221805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:03.222008 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:03.222141 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:03.222910 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:03.223681 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:03.224557 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:03.224970 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:03.225054 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:03.227685 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:03.227695 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:03.227709 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:03.227734 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:03.229483 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:03.229943 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:03.229948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:03.229950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:03.230030 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:03.230283 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:03.230285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:03.230287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:03.230518 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:03.230714 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:03.230956 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 02:47:03.231153 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":139,"threads_started":1}
Aug 11 02:47:03 dist-test-slave-n326 krb5kdc[28489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:03.243780 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:03.234497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55492 (local address 127.21.236.190:39987)
0811 02:47:03.234652 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:03.234655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:03.234697 (+ 42us) server_negotiation.cc:407] Connection header received
0811 02:47:03.234813 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:03.234815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:03.234856 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:03.234925 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:03.235599 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:03.236200 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:03.237137 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:03.237403 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:03.237513 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:03.240366 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:03.240370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:03.240372 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:03.240399 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:03.241945 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:03.242360 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:03.242363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:03.242364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:03.242444 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:03.242742 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:03.242744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:03.242745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:03.243014 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:03.243244 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:03.243418 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 02:47:03.243552 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 02:47:03.244278 28572 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55492
I20260811 02:47:03.245065 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 28691
I20260811 02:47:03.257726 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 28926
I20260811 02:47:03.286538 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 29142
I20260811 02:47:03.312520 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 28511
2026-08-11T02:47:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2694 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:47:03 dist-test-slave-n326 krb5kdc[29391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:47:03 dist-test-slave-n326 krb5kdc[29391](info): set up 2 sockets
Aug 11 02:47:03 dist-test-slave-n326 krb5kdc[29391](info): commencing operation
krb5kdc: starting...
W20260811 02:47:04.708976 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.368s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:47:04 dist-test-slave-n326 krb5kdc[29391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:47:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:47:04.851142 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:46075
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:46075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:04.968763 29421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:04.969105 29421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:04.969180 29421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:04.973313 29421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:04.973414 29421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:04.973472 29421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:04.973533 29421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:04.973570 29421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:47:04.978883 29421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:46075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:46075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:04.980032 29421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:04.981102 29421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:04.989965 29433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:04.989964 29435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:04.989959 29432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:04.990399 29421 server_base.cc:1034] running on GCE node
I20260811 02:47:04.991262 29421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:04.992605 29421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:04.993806 29421 hybrid_clock.cc:648] HybridClock initialized: now 1786416424993774 us; error 44 us; skew 500 ppm
Aug 11 02:47:04 dist-test-slave-n326 krb5kdc[29391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416424, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:04.997098 29421 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:47:04.997916 29421 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:04.997989 29421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:04.998227 29421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:05.000624 29421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "932ee93b81874869b5a254fc5c87d913"
format_stamp: "Formatted at 2026-08-11 02:47:04 on dist-test-slave-n326"
server_key: "f5ef9be5ee35daa0a48ae3322504cf41"
server_key_iv: "3a8cf5b1ddca407d2398d07f976734bd"
server_key_version: "encryptionkey@0"
I20260811 02:47:05.001245 29421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "932ee93b81874869b5a254fc5c87d913"
format_stamp: "Formatted at 2026-08-11 02:47:04 on dist-test-slave-n326"
server_key: "f5ef9be5ee35daa0a48ae3322504cf41"
server_key_iv: "3a8cf5b1ddca407d2398d07f976734bd"
server_key_version: "encryptionkey@0"
I20260811 02:47:05.005383 29421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:47:05.008136 29443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:05.009443 29421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:47:05.009613 29421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "932ee93b81874869b5a254fc5c87d913"
format_stamp: "Formatted at 2026-08-11 02:47:04 on dist-test-slave-n326"
server_key: "f5ef9be5ee35daa0a48ae3322504cf41"
server_key_iv: "3a8cf5b1ddca407d2398d07f976734bd"
server_key_version: "encryptionkey@0"
I20260811 02:47:05.009720 29421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:05.026329 29421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:05.030002 29421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:05.030191 29421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:05.038233 29421 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:46075
I20260811 02:47:05.038589 29531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:46075 every 8 connection(s)
I20260811 02:47:05.039019 29421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:05.039625 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 29421
I20260811 02:47:05.039779 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:47:05.040141 22450 external_mini_cluster.cc:1428] Setting key dfc5b1cfc41ff08a8ea0c9180f2ee56b
I20260811 02:47:05.042532 29532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:05.048250 29532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913: Bootstrap starting.
Aug 11 02:47:05 dist-test-slave-n326 krb5kdc[29391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:05.050586 29532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:05.051415 29532 log.cc:824] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:05.053865 29532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913: No bootstrap required, opened a new log
I20260811 02:47:05.056064 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.041656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45094 (local address 127.21.236.190:46075)
0811 02:47:05.042110 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:05.042127 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:05.042166 (+ 39us) server_negotiation.cc:407] Connection header received
0811 02:47:05.042948 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:05.042968 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:05.043384 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:05.043754 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:05.044666 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.045752 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.046725 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.047328 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.047515 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.050341 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:05.050346 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:05.050361 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:05.050390 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:05.052547 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:05.053021 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:05.053029 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:05.053035 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:05.053183 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:05.053471 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:05.053474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:05.053475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:05.054004 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:05.054183 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:05.054492 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 02:47:05.054799 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":135,"threads_started":1}
I20260811 02:47:05.057848 29532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "932ee93b81874869b5a254fc5c87d913" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46075 } }
I20260811 02:47:05.058073 29532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:05.058110 29532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 932ee93b81874869b5a254fc5c87d913, State: Initialized, Role: FOLLOWER
I20260811 02:47:05.058590 29532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "932ee93b81874869b5a254fc5c87d913" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46075 } }
I20260811 02:47:05.058719 29532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:05.058759 29532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:05.058831 29532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:05.059794 29532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "932ee93b81874869b5a254fc5c87d913" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46075 } }
I20260811 02:47:05.060176 29532 leader_election.cc:302] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 932ee93b81874869b5a254fc5c87d913; no voters:
I20260811 02:47:05.060568 29532 leader_election.cc:288] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:47:05.060720 29544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:05.060990 29544 raft_consensus.cc:686] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [term 1 LEADER]: Becoming Leader. State: Replica: 932ee93b81874869b5a254fc5c87d913, State: Running, Role: LEADER
I20260811 02:47:05.061306 29544 consensus_queue.cc:237] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "932ee93b81874869b5a254fc5c87d913" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46075 } }
I20260811 02:47:05.061897 29532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:05.063099 29546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "932ee93b81874869b5a254fc5c87d913" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "932ee93b81874869b5a254fc5c87d913" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46075 } } }
I20260811 02:47:05.063251 29546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:05.063581 29545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 932ee93b81874869b5a254fc5c87d913. Latest consensus state: current_term: 1 leader_uuid: "932ee93b81874869b5a254fc5c87d913" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "932ee93b81874869b5a254fc5c87d913" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46075 } } }
I20260811 02:47:05.063684 29545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:05.064529 29557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:05.067932 29557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:05.074250 29557 catalog_manager.cc:1347] Generated new cluster ID: ba65f7bc73bc4ecea9489da4639fd662
I20260811 02:47:05.074338 29557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:05.095306 29557 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:47:05.096329 29557 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:05.104075 29557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913: Generated new TSK 0
I20260811 02:47:05.104835 29557 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:47:05.147590 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:46075
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:05.265316 29583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:05.265602 29583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:05.265650 29583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:05.269764 29583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:05.269831 29583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:05.269892 29583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:47:05.275367 29583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:46075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:05.276458 29583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:05.277565 29583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:05.287400 29594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:05.287400 29589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:05.287473 29590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:05.287781 29583 server_base.cc:1034] running on GCE node
I20260811 02:47:05.288369 29583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:05.289006 29583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:05.290162 29583 hybrid_clock.cc:648] HybridClock initialized: now 1786416425290142 us; error 33 us; skew 500 ppm
Aug 11 02:47:05 dist-test-slave-n326 krb5kdc[29391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416425, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:05.293365 29583 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:47:05.294185 29583 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:05.294250 29583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:05.294483 29583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:05.296666 29583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "bf8536951baf420695e681df22d096ba"
format_stamp: "Formatted at 2026-08-11 02:47:05 on dist-test-slave-n326"
server_key: "ee501ac85dedddea06439e1563c891e3"
server_key_iv: "56d31e533ac9b725f71ec79b06262d0d"
server_key_version: "encryptionkey@0"
I20260811 02:47:05.297245 29583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "bf8536951baf420695e681df22d096ba"
format_stamp: "Formatted at 2026-08-11 02:47:05 on dist-test-slave-n326"
server_key: "ee501ac85dedddea06439e1563c891e3"
server_key_iv: "56d31e533ac9b725f71ec79b06262d0d"
server_key_version: "encryptionkey@0"
I20260811 02:47:05.301139 29583 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:47:05.303812 29605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:05.305138 29583 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:47:05.305271 29583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "bf8536951baf420695e681df22d096ba"
format_stamp: "Formatted at 2026-08-11 02:47:05 on dist-test-slave-n326"
server_key: "ee501ac85dedddea06439e1563c891e3"
server_key_iv: "56d31e533ac9b725f71ec79b06262d0d"
server_key_version: "encryptionkey@0"
I20260811 02:47:05.305382 29583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:05.318374 29583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:05.321677 29583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:05.321928 29583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:05.322567 29583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:05.323545 29583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:05.323621 29583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:05.323696 29583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:05.323746 29583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:05.333750 29583 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:45501
I20260811 02:47:05.333768 29805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:45501 every 8 connection(s)
I20260811 02:47:05.334389 29583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:47:05.335736 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 29583
I20260811 02:47:05.335876 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:47:05.336207 22450 external_mini_cluster.cc:1428] Setting key c47a30e277c7f7c02c69b43f49e2bbc9
Aug 11 02:47:05 dist-test-slave-n326 krb5kdc[29391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416425, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
I20260811 02:47:05.351326 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.336584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:53295 (local address 127.21.236.190:46075)
0811 02:47:05.336761 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:05.336766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:05.337870 (+ 1104us) server_negotiation.cc:407] Connection header received
0811 02:47:05.339428 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:05.339434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:05.339497 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:05.339592 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:05.341387 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.341939 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.344039 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.344269 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.344355 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.347536 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:05.347539 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:05.347542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:05.347565 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:05.349155 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:05.349695 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:05.349699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:05.349701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:05.349788 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:05.350152 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:05.350155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:05.350157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:05.350342 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:05.350461 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:05.350990 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 02:47:05.351149 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
I20260811 02:47:05.352396 29812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.337109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:46075 (local address 127.21.236.129:53295)
0811 02:47:05.337673 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:05.337757 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:05.339212 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:05.339767 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:05.339775 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:05.340406 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:05.341189 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.341213 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.342086 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.342093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.343903 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.343918 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.344449 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.344457 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.344723 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.345435 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:05.345466 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:05.347361 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:05.349286 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:05.349293 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:05.349308 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:05.349583 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:05.349882 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:05.349886 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:05.349888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:05.350047 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:05.350461 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:05.350469 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:05.350715 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 02:47:05.350976 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 02:47:05.351338 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":130,"threads_started":1}
I20260811 02:47:05.353832 29808 heartbeater.cc:347] Connected to a master server at 127.21.236.190:46075
I20260811 02:47:05.354156 29808 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:05.354784 29808 heartbeater.cc:511] Master 127.21.236.190:46075 requested a full tablet report, sending...
I20260811 02:47:05.356473 29468 ts_manager.cc:194] Registered new tserver with Master: bf8536951baf420695e681df22d096ba (127.21.236.129:45501)
I20260811 02:47:05.357677 29468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:53295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:47:05.370671 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:46075
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:05.489753 29816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:05.490103 29816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:05.490224 29816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:05.494411 29816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:05.494513 29816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:05.494606 29816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:47:05.500157 29816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:46075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:05.501324 29816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:05.502417 29816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:05.512302 29828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:05.512318 29825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:05.512507 29816 server_base.cc:1034] running on GCE node
W20260811 02:47:05.512707 29822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:05.513127 29816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:05.513872 29816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:05.515189 29816 hybrid_clock.cc:648] HybridClock initialized: now 1786416425515136 us; error 62 us; skew 500 ppm
Aug 11 02:47:05 dist-test-slave-n326 krb5kdc[29391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416425, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:05.518435 29816 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:47:05.519388 29816 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:05.519485 29816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:05.519732 29816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:05.521873 29816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "06cf9e1e37c4485b81b7010f83479ae1"
format_stamp: "Formatted at 2026-08-11 02:47:05 on dist-test-slave-n326"
server_key: "6ea60ea6470f185f24aee01454cd3557"
server_key_iv: "7f4b886bb77231c334c0732a06cb97f7"
server_key_version: "encryptionkey@0"
I20260811 02:47:05.522454 29816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "06cf9e1e37c4485b81b7010f83479ae1"
format_stamp: "Formatted at 2026-08-11 02:47:05 on dist-test-slave-n326"
server_key: "6ea60ea6470f185f24aee01454cd3557"
server_key_iv: "7f4b886bb77231c334c0732a06cb97f7"
server_key_version: "encryptionkey@0"
I20260811 02:47:05.526324 29816 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:47:05.529053 29836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:05.530279 29816 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:47:05.530414 29816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "06cf9e1e37c4485b81b7010f83479ae1"
format_stamp: "Formatted at 2026-08-11 02:47:05 on dist-test-slave-n326"
server_key: "6ea60ea6470f185f24aee01454cd3557"
server_key_iv: "7f4b886bb77231c334c0732a06cb97f7"
server_key_version: "encryptionkey@0"
I20260811 02:47:05.530557 29816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:05.547369 29816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:05.550771 29816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:05.550974 29816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:05.551640 29816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:05.552561 29816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:05.552629 29816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:05.552700 29816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:05.552740 29816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:05.562688 29816 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:41531
I20260811 02:47:05.562721 30029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:41531 every 8 connection(s)
I20260811 02:47:05.563623 29816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:47:05.568452 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 29816
I20260811 02:47:05.568567 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:47:05.568924 22450 external_mini_cluster.cc:1428] Setting key 448c248c6d2532750e84ca3e7ee71f7d
Aug 11 02:47:05 dist-test-slave-n326 krb5kdc[29391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416425, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:05.579082 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.565488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:59225 (local address 127.21.236.190:46075)
0811 02:47:05.565638 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:05.565642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:05.566528 (+ 886us) server_negotiation.cc:407] Connection header received
0811 02:47:05.567939 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:05.567942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:05.567992 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:05.568073 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:05.569521 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.570157 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.571490 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.571885 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.572036 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.574971 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:05.574978 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:05.574984 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:05.575019 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:05.576780 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:05.577294 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:05.577300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:05.577305 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:05.577408 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:05.577771 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:05.577777 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:05.577780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:05.578037 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:05.578155 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:05.578699 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 02:47:05.578853 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:47:05.579934 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.565915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:46075 (local address 127.21.236.130:59225)
0811 02:47:05.566372 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:05.566416 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:05.567706 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:05.568241 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:05.568248 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:05.568748 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:05.569330 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.569351 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.570292 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.570295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.571364 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.571375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.572134 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.572137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.572308 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.573011 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:05.573028 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:05.574789 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:05.576906 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:05.576911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:05.576924 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:05.577186 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:05.577524 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:05.577526 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:05.577528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:05.577668 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:05.578172 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:05.578178 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:05.578434 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 02:47:05.578608 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 02:47:05.578897 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
I20260811 02:47:05.581274 30031 heartbeater.cc:347] Connected to a master server at 127.21.236.190:46075
I20260811 02:47:05.581578 30031 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:05.582091 30031 heartbeater.cc:511] Master 127.21.236.190:46075 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
I20260811 02:47:05.583330 29468 ts_manager.cc:194] Registered new tserver with Master: 06cf9e1e37c4485b81b7010f83479ae1 (127.21.236.130:41531)
I20260811 02:47:05.583971 29468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:59225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:47:05.602723 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:46075
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:05.723976 30044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:05.724270 30044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:05.724319 30044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:05.728473 30044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:05.728554 30044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:05.728616 30044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:47:05.734246 30044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:46075
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:05.735424 30044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:05.736518 30044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:05.746915 30060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:05.746857 30057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:05.746865 30058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:05.747203 30044 server_base.cc:1034] running on GCE node
I20260811 02:47:05.747807 30044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:05.748513 30044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:05.749748 30044 hybrid_clock.cc:648] HybridClock initialized: now 1786416425749704 us; error 57 us; skew 500 ppm
Aug 11 02:47:05 dist-test-slave-n326 krb5kdc[29391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416425, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:05.753225 30044 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:47:05.754093 30044 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:05.754180 30044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:05.754429 30044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:05.757654 30044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "8e91b59730694d93a7120aa047c102b2"
format_stamp: "Formatted at 2026-08-11 02:47:05 on dist-test-slave-n326"
server_key: "359565264ee1add82b91d97d8d325b68"
server_key_iv: "67fbbc5ed9bc11e89f6e19b5ca6e107b"
server_key_version: "encryptionkey@0"
I20260811 02:47:05.758351 30044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "8e91b59730694d93a7120aa047c102b2"
format_stamp: "Formatted at 2026-08-11 02:47:05 on dist-test-slave-n326"
server_key: "359565264ee1add82b91d97d8d325b68"
server_key_iv: "67fbbc5ed9bc11e89f6e19b5ca6e107b"
server_key_version: "encryptionkey@0"
I20260811 02:47:05.762956 30044 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.003s
I20260811 02:47:05.766011 30071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:05.767555 30044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:47:05.767717 30044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "8e91b59730694d93a7120aa047c102b2"
format_stamp: "Formatted at 2026-08-11 02:47:05 on dist-test-slave-n326"
server_key: "359565264ee1add82b91d97d8d325b68"
server_key_iv: "67fbbc5ed9bc11e89f6e19b5ca6e107b"
server_key_version: "encryptionkey@0"
I20260811 02:47:05.767841 30044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:05.783804 30044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:05.787352 30044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:05.787560 30044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:05.788196 30044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:05.789103 30044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:05.789157 30044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:05.789198 30044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:05.789237 30044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:05.799268 30044 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:36203
I20260811 02:47:05.799286 30260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:36203 every 8 connection(s)
I20260811 02:47:05.799906 30044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:47:05.801038 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 30044
I20260811 02:47:05.801177 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:47:05.801484 22450 external_mini_cluster.cc:1428] Setting key 1fbf4f0c64cb87f201bbf357a7187142
Aug 11 02:47:05 dist-test-slave-n326 krb5kdc[29391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416425, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:05.815248 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.801830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:33059 (local address 127.21.236.190:46075)
0811 02:47:05.801989 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:05.801996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:05.803125 (+ 1129us) server_negotiation.cc:407] Connection header received
0811 02:47:05.804847 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:05.804850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:05.804934 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:05.805023 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:05.806276 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.806885 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.808087 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.808327 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.808429 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.811459 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:05.811462 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:05.811464 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:05.811491 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:05.812873 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:05.813433 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:05.813436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:05.813438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:05.813523 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:05.813846 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:05.813848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:05.813850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:05.814144 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:05.814264 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:05.814903 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 02:47:05.815044 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:47:05.816097 30266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.802348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:46075 (local address 127.21.236.131:33059)
0811 02:47:05.802947 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:05.803005 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:05.804622 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:05.805229 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:05.805236 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:05.805615 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:05.806138 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.806151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.807015 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.807018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.807945 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.807955 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.808642 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.808645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.808797 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.809551 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:05.809569 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:05.811253 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:05.813032 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:05.813044 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:05.813057 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:05.813312 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:05.813608 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:05.813611 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:05.813612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:05.813751 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:05.814271 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:05.814278 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:05.814648 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 02:47:05.814841 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 02:47:05.815207 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":157,"threads_started":1}
I20260811 02:47:05.817250 30263 heartbeater.cc:347] Connected to a master server at 127.21.236.190:46075
I20260811 02:47:05.817505 30263 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:05.818111 30263 heartbeater.cc:511] Master 127.21.236.190:46075 requested a full tablet report, sending...
I20260811 02:47:05.819305 29468 ts_manager.cc:194] Registered new tserver with Master: 8e91b59730694d93a7120aa047c102b2 (127.21.236.131:36203)
I20260811 02:47:05.819939 29468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:33059
I20260811 02:47:05.828071 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:47:05.838886 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.830279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45098 (local address 127.21.236.190:46075)
0811 02:47:05.830460 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:05.830467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:05.830613 (+ 146us) server_negotiation.cc:407] Connection header received
0811 02:47:05.830723 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:05.830725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:05.830771 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:05.830845 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:05.831653 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.832213 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.833154 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.833446 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.833560 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.835038 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:05.835042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:05.835045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:05.835121 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:05.837084 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:05.837497 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:05.837500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:05.837501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:05.837609 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:05.837888 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:05.837892 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:05.837893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:05.838157 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:05.838282 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:05.838524 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 02:47:05.838676 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 02:47:05.842080 29468 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:47:05.845443 29468 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:47:05.861055 30284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.854793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:45501 (local address 127.0.0.1:38308)
0811 02:47:05.855619 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:05.855672 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:05.855792 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:05.857210 (+ 1418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:05.857217 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:05.857242 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:05.857349 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.857358 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.858668 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.858672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.859637 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.859647 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.860473 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.860477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.860586 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.860623 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 02:47:05.860764 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 02:47:05.860890 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":61,"thread_start_us":82,"threads_started":1}
I20260811 02:47:05.861066 30286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.854905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38308 (local address 127.21.236.129:45501)
0811 02:47:05.856732 (+ 1827us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:05.856739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:05.856761 (+ 22us) server_negotiation.cc:407] Connection header received
0811 02:47:05.856829 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:05.856834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:05.857031 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:05.857211 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:05.857477 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.858543 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.859766 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.860339 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.860477 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.860625 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:05.860795 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 02:47:05.860897 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1708,"thread_start_us":91,"threads_started":1}
I20260811 02:47:05.861403 30288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.854918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:41531 (local address 127.0.0.1:32916)
0811 02:47:05.855801 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:05.855819 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:05.855938 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:05.856739 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:05.856746 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:05.856771 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:05.856914 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.856921 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.858454 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.858458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.859679 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.859691 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.860908 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.860912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.861051 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.861093 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 02:47:05.861150 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 02:47:05.861227 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"mutex_wait_us":43,"thread_start_us":147,"threads_started":1}
I20260811 02:47:05.862092 30289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.855693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32916 (local address 127.21.236.130:41531)
0811 02:47:05.856163 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:05.856172 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:05.856192 (+ 20us) server_negotiation.cc:407] Connection header received
0811 02:47:05.856299 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:05.856304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:05.856557 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:05.856733 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:05.857060 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.858292 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.859839 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.860755 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.861547 (+ 792us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.861688 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:05.861745 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 02:47:05.861846 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":114,"threads_started":1}
I20260811 02:47:05.862658 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.855251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:36203 (local address 127.0.0.1:49988)
0811 02:47:05.856417 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:05.856428 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:05.856517 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:05.856971 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:05.856974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:05.856995 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:05.857127 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.857136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.858446 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.858450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.861008 (+ 2558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.861018 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.862298 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.862301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.862385 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.862410 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 02:47:05.862472 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 02:47:05.862536 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1114,"mutex_wait_us":75,"thread_start_us":104,"threads_started":1}
I20260811 02:47:05.863505 30290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.856089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49988 (local address 127.21.236.131:36203)
0811 02:47:05.856434 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:05.856451 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:05.856525 (+ 74us) server_negotiation.cc:407] Connection header received
0811 02:47:05.856605 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:05.856612 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:05.856815 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:05.856992 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:05.857300 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.858270 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.861155 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.862163 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.862304 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.863244 (+ 940us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:05.863292 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 02:47:05.863367 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":112,"threads_started":1}
I20260811 02:47:05.865386 29688 tablet_service.cc:1467] Processing CreateTablet for tablet ac3905769071424f82d03b60ff9ff754 (DEFAULT_TABLE table=test-table [id=8b4a2d9eaa034102a003320fdcfb8527]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:05.865389 29916 tablet_service.cc:1467] Processing CreateTablet for tablet ac3905769071424f82d03b60ff9ff754 (DEFAULT_TABLE table=test-table [id=8b4a2d9eaa034102a003320fdcfb8527]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:05.866583 29916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac3905769071424f82d03b60ff9ff754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:05.866583 29688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac3905769071424f82d03b60ff9ff754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:05.866179 30151 tablet_service.cc:1467] Processing CreateTablet for tablet ac3905769071424f82d03b60ff9ff754 (DEFAULT_TABLE table=test-table [id=8b4a2d9eaa034102a003320fdcfb8527]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:05.867331 30151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac3905769071424f82d03b60ff9ff754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:05.874013 30292 tablet_bootstrap.cc:492] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2: Bootstrap starting.
I20260811 02:47:05.875169 30293 tablet_bootstrap.cc:492] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba: Bootstrap starting.
I20260811 02:47:05.875279 30298 tablet_bootstrap.cc:492] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1: Bootstrap starting.
I20260811 02:47:05.877269 30298 tablet_bootstrap.cc:654] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:05.877388 30293 tablet_bootstrap.cc:654] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:05.878194 30293 log.cc:824] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:05.878363 30298 log.cc:824] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:05.880268 30298 tablet_bootstrap.cc:492] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1: No bootstrap required, opened a new log
I20260811 02:47:05.880394 30293 tablet_bootstrap.cc:492] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba: No bootstrap required, opened a new log
I20260811 02:47:05.880596 30298 ts_tablet_manager.cc:1397] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:47:05.880595 30293 ts_tablet_manager.cc:1397] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:47:05.880791 30292 tablet_bootstrap.cc:654] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:05.882125 30292 log.cc:824] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:05.884548 30292 tablet_bootstrap.cc:492] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2: No bootstrap required, opened a new log
I20260811 02:47:05.884755 30292 ts_tablet_manager.cc:1397] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260811 02:47:05.884519 30298 raft_consensus.cc:348] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8536951baf420695e681df22d096ba" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45501 } } peers { permanent_uuid: "06cf9e1e37c4485b81b7010f83479ae1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41531 } } peers { permanent_uuid: "8e91b59730694d93a7120aa047c102b2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36203 } }
I20260811 02:47:05.884855 30298 raft_consensus.cc:374] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:05.884927 30298 raft_consensus.cc:729] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06cf9e1e37c4485b81b7010f83479ae1, State: Initialized, Role: FOLLOWER
I20260811 02:47:05.885206 30293 raft_consensus.cc:348] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8536951baf420695e681df22d096ba" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45501 } } peers { permanent_uuid: "06cf9e1e37c4485b81b7010f83479ae1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41531 } } peers { permanent_uuid: "8e91b59730694d93a7120aa047c102b2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36203 } }
I20260811 02:47:05.885460 30293 raft_consensus.cc:374] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:05.885520 30293 raft_consensus.cc:729] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf8536951baf420695e681df22d096ba, State: Initialized, Role: FOLLOWER
I20260811 02:47:05.885439 30298 consensus_queue.cc:260] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8536951baf420695e681df22d096ba" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45501 } } peers { permanent_uuid: "06cf9e1e37c4485b81b7010f83479ae1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41531 } } peers { permanent_uuid: "8e91b59730694d93a7120aa047c102b2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36203 } }
I20260811 02:47:05.885934 30293 consensus_queue.cc:260] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8536951baf420695e681df22d096ba" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45501 } } peers { permanent_uuid: "06cf9e1e37c4485b81b7010f83479ae1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41531 } } peers { permanent_uuid: "8e91b59730694d93a7120aa047c102b2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36203 } }
I20260811 02:47:05.886292 30031 heartbeater.cc:503] Master 127.21.236.190:46075 was elected leader, sending a full tablet report...
I20260811 02:47:05.886646 29808 heartbeater.cc:503] Master 127.21.236.190:46075 was elected leader, sending a full tablet report...
I20260811 02:47:05.886646 30298 ts_tablet_manager.cc:1428] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:47:05.887542 30293 ts_tablet_manager.cc:1428] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 02:47:05.890038 30292 raft_consensus.cc:348] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8536951baf420695e681df22d096ba" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45501 } } peers { permanent_uuid: "06cf9e1e37c4485b81b7010f83479ae1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41531 } } peers { permanent_uuid: "8e91b59730694d93a7120aa047c102b2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36203 } }
I20260811 02:47:05.890328 30292 raft_consensus.cc:374] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:05.890398 30292 raft_consensus.cc:729] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e91b59730694d93a7120aa047c102b2, State: Initialized, Role: FOLLOWER
I20260811 02:47:05.890864 30292 consensus_queue.cc:260] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8536951baf420695e681df22d096ba" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45501 } } peers { permanent_uuid: "06cf9e1e37c4485b81b7010f83479ae1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41531 } } peers { permanent_uuid: "8e91b59730694d93a7120aa047c102b2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36203 } }
I20260811 02:47:05.891707 30263 heartbeater.cc:503] Master 127.21.236.190:46075 was elected leader, sending a full tablet report...
I20260811 02:47:05.892282 30292 ts_tablet_manager.cc:1428] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 02:47:05.895573 30303 raft_consensus.cc:482] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:47:05.895707 30303 raft_consensus.cc:504] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8536951baf420695e681df22d096ba" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45501 } } peers { permanent_uuid: "06cf9e1e37c4485b81b7010f83479ae1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41531 } } peers { permanent_uuid: "8e91b59730694d93a7120aa047c102b2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36203 } }
I20260811 02:47:05.896642 30303 leader_election.cc:288] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf8536951baf420695e681df22d096ba (127.21.236.129:45501), 8e91b59730694d93a7120aa047c102b2 (127.21.236.131:36203)
I20260811 02:47:05.900807 30035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.896909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:36203 (local address 127.21.236.130:57493)
0811 02:47:05.897075 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:05.897097 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:05.897172 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:05.897506 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:05.897508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:05.897526 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:05.897638 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.897642 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.898547 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.898549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.899790 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.899802 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.900465 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.900468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.900555 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.900577 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:47:05.900644 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:47:05.900700 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 02:47:05.900854 30290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.896998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:57493 (local address 127.21.236.131:36203)
0811 02:47:05.897308 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:05.897312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:05.897323 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:47:05.897360 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:05.897362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:05.897396 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:05.897466 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:05.897757 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.898448 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.899916 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.900340 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.900499 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.900551 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:05.900675 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 02:47:05.900742 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236}
I20260811 02:47:05.901605 30307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.896908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:45501 (local address 127.21.236.130:59957)
0811 02:47:05.897860 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:05.897873 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:05.897968 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:05.898177 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:05.898179 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:05.898192 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:05.898295 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.898299 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.899045 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.899048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.900223 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:05.900232 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.901251 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.901258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:05.901370 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.901388 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 02:47:05.901453 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 02:47:05.901502 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":618,"thread_start_us":131,"threads_started":1}
I20260811 02:47:05.901574 30179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac3905769071424f82d03b60ff9ff754" candidate_uuid: "06cf9e1e37c4485b81b7010f83479ae1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e91b59730694d93a7120aa047c102b2" is_pre_election: true
I20260811 02:47:05.901767 30286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.897074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:59957 (local address 127.21.236.129:45501)
0811 02:47:05.897180 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:05.897184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:05.897966 (+ 782us) server_negotiation.cc:407] Connection header received
0811 02:47:05.898039 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:05.898042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:05.898084 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:05.898160 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:05.898395 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.898923 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.900341 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.900971 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.901125 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.901174 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:05.901528 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 02:47:05.901625 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 02:47:05.901876 30179 raft_consensus.cc:2454] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06cf9e1e37c4485b81b7010f83479ae1 in term 0.
I20260811 02:47:05.902586 29847 leader_election.cc:302] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06cf9e1e37c4485b81b7010f83479ae1, 8e91b59730694d93a7120aa047c102b2; no voters:
I20260811 02:47:05.902616 29718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac3905769071424f82d03b60ff9ff754" candidate_uuid: "06cf9e1e37c4485b81b7010f83479ae1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf8536951baf420695e681df22d096ba" is_pre_election: true
I20260811 02:47:05.902891 29718 raft_consensus.cc:2454] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06cf9e1e37c4485b81b7010f83479ae1 in term 0.
I20260811 02:47:05.902894 30303 raft_consensus.cc:2781] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:47:05.902987 30303 raft_consensus.cc:482] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:47:05.903034 30303 raft_consensus.cc:3037] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:05.904160 30303 raft_consensus.cc:504] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8536951baf420695e681df22d096ba" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45501 } } peers { permanent_uuid: "06cf9e1e37c4485b81b7010f83479ae1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41531 } } peers { permanent_uuid: "8e91b59730694d93a7120aa047c102b2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36203 } }
I20260811 02:47:05.904459 30303 leader_election.cc:288] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [CANDIDATE]: Term 1 election: Requested vote from peers bf8536951baf420695e681df22d096ba (127.21.236.129:45501), 8e91b59730694d93a7120aa047c102b2 (127.21.236.131:36203)
I20260811 02:47:05.904853 29718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac3905769071424f82d03b60ff9ff754" candidate_uuid: "06cf9e1e37c4485b81b7010f83479ae1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf8536951baf420695e681df22d096ba"
I20260811 02:47:05.904851 30179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac3905769071424f82d03b60ff9ff754" candidate_uuid: "06cf9e1e37c4485b81b7010f83479ae1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e91b59730694d93a7120aa047c102b2"
I20260811 02:47:05.905005 30179 raft_consensus.cc:3037] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:05.905006 29718 raft_consensus.cc:3037] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:05.906471 30179 raft_consensus.cc:2454] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06cf9e1e37c4485b81b7010f83479ae1 in term 1.
I20260811 02:47:05.906476 29718 raft_consensus.cc:2454] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06cf9e1e37c4485b81b7010f83479ae1 in term 1.
I20260811 02:47:05.906795 29847 leader_election.cc:302] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06cf9e1e37c4485b81b7010f83479ae1, 8e91b59730694d93a7120aa047c102b2; no voters:
I20260811 02:47:05.907109 30303 raft_consensus.cc:2781] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:05.907354 30303 raft_consensus.cc:686] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [term 1 LEADER]: Becoming Leader. State: Replica: 06cf9e1e37c4485b81b7010f83479ae1, State: Running, Role: LEADER
I20260811 02:47:05.907721 30303 consensus_queue.cc:237] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8536951baf420695e681df22d096ba" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45501 } } peers { permanent_uuid: "06cf9e1e37c4485b81b7010f83479ae1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41531 } } peers { permanent_uuid: "8e91b59730694d93a7120aa047c102b2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36203 } }
I20260811 02:47:05.911154 29466 catalog_manager.cc:5495] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06cf9e1e37c4485b81b7010f83479ae1 (127.21.236.130). New cstate: current_term: 1 leader_uuid: "06cf9e1e37c4485b81b7010f83479ae1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf8536951baf420695e681df22d096ba" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06cf9e1e37c4485b81b7010f83479ae1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e91b59730694d93a7120aa047c102b2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36203 } health_report { overall_health: UNKNOWN } } }
I20260811 02:47:05.924233 30289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.920543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32924 (local address 127.21.236.130:41531)
0811 02:47:05.920678 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:05.920682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:05.920701 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:47:05.920804 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:05.920806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:05.920858 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:05.920973 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:47:05.921270 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.921863 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.922747 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:05.922995 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:05.923162 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:05.923430 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:47:05.923834 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:47:05.923946 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:05.924013 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 02:47:05.924062 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:47:05.930346 29718 raft_consensus.cc:1261] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba [term 1 FOLLOWER]: Refusing update from remote peer 06cf9e1e37c4485b81b7010f83479ae1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:05.930392 30179 raft_consensus.cc:1261] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2 [term 1 FOLLOWER]: Refusing update from remote peer 06cf9e1e37c4485b81b7010f83479ae1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:05.931118 30303 consensus_queue.cc:1035] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf8536951baf420695e681df22d096ba" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:05.931308 30308 consensus_queue.cc:1035] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e91b59730694d93a7120aa047c102b2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:05.938958 30311 mvcc.cc:205] Tried to move back new op lower bound from 7317161680602877952 to 7317161680520683520. Current Snapshot: MvccSnapshot[applied={T|T < 7317161680602877952}]
I20260811 02:47:05.950006 29466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:05.950215 29466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:05.952806 29466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 932ee93b81874869b5a254fc5c87d913: Sending DeleteTablet for 3 replicas of tablet ac3905769071424f82d03b60ff9ff754
I20260811 02:47:05.953661 29916 tablet_service.cc:1514] Processing DeleteTablet for tablet ac3905769071424f82d03b60ff9ff754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:05 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:32916
I20260811 02:47:05.953727 30151 tablet_service.cc:1514] Processing DeleteTablet for tablet ac3905769071424f82d03b60ff9ff754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:05 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:49988
I20260811 02:47:05.953866 29688 tablet_service.cc:1514] Processing DeleteTablet for tablet ac3905769071424f82d03b60ff9ff754 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:05 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:38308
I20260811 02:47:05.954310 30323 tablet_replica.cc:330] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2: stopping tablet replica
I20260811 02:47:05.954310 30322 tablet_replica.cc:330] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1: stopping tablet replica
I20260811 02:47:05.954679 30322 raft_consensus.cc:2229] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:47:05.954903 30325 tablet_replica.cc:330] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba: stopping tablet replica
I20260811 02:47:05.955180 30322 raft_consensus.cc:2258] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:05.955334 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 29583
I20260811 02:47:05.955319 30325 raft_consensus.cc:2229] T ac3905769071424f82d03b60ff9ff754 P bf8536951baf420695e681df22d096ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:05.960011 30322 ts_tablet_manager.cc:1905] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:05.963687 30322 ts_tablet_manager.cc:1918] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:05.963809 30322 log.cc:1196] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/ac3905769071424f82d03b60ff9ff754
I20260811 02:47:05.964192 30322 ts_tablet_manager.cc:1939] T ac3905769071424f82d03b60ff9ff754 P 06cf9e1e37c4485b81b7010f83479ae1: Deleting consensus metadata
I20260811 02:47:05.965674 29445 catalog_manager.cc:4841] TS 06cf9e1e37c4485b81b7010f83479ae1 (127.21.236.130:41531): tablet ac3905769071424f82d03b60ff9ff754 (table test-table [id=8b4a2d9eaa034102a003320fdcfb8527]) successfully deleted
I20260811 02:47:05.969260 30323 raft_consensus.cc:2229] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:05.969697 30323 raft_consensus.cc:2258] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:05.971654 30323 ts_tablet_manager.cc:1905] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:05.974768 30323 ts_tablet_manager.cc:1918] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:05.974885 30323 log.cc:1196] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/ac3905769071424f82d03b60ff9ff754
W20260811 02:47:05.975126 29444 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:45501 (error 108)
I20260811 02:47:05.975349 30323 ts_tablet_manager.cc:1939] T ac3905769071424f82d03b60ff9ff754 P 8e91b59730694d93a7120aa047c102b2: Deleting consensus metadata
I20260811 02:47:05.976293 30284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:05.975951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:45501 (local address 127.0.0.1:38314)
0811 02:47:05.976093 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:05.976183 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:45501: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 02:47:05.976373 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 29816
I20260811 02:47:05.976734 29447 catalog_manager.cc:4841] TS 8e91b59730694d93a7120aa047c102b2 (127.21.236.131:36203): tablet ac3905769071424f82d03b60ff9ff754 (table test-table [id=8b4a2d9eaa034102a003320fdcfb8527]) successfully deleted
W20260811 02:47:05.979266 29444 catalog_manager.cc:4584] TS bf8536951baf420695e681df22d096ba (127.21.236.129:45501): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ac3905769071424f82d03b60ff9ff754: Network error: Client connection negotiation failed: client connection to 127.21.236.129:45501: connect: Connection refused (error 111)
I20260811 02:47:05.990377 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 30044
I20260811 02:47:06.007310 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 29421
2026-08-11T02:47:06Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2696 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T02:47:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:06Z Disabled control of system clock
I20260811 02:47:06.046151 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:35885
--webserver_interface=127.21.236.190
--webserver_port=0
--builtin_ntp_servers=127.21.236.148:45579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:35885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 02:47:06.160073 30337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:06.160337 30337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:06.160374 30337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:06.164350 30337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:06.164409 30337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:06.164425 30337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:06.164443 30337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:06.164458 30337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:47:06.169188 30337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:35885
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:35885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:06.170265 30337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:06.171686 30337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:06.180259 30349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:06.180389 30351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:06.180599 30337 server_base.cc:1034] running on GCE node
W20260811 02:47:06.180249 30353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:06.181354 30337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:06.182665 30337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:06.183852 30337 hybrid_clock.cc:648] HybridClock initialized: now 1786416426183805 us; error 62 us; skew 500 ppm
I20260811 02:47:06.185971 30337 webserver.cc:460] Webserver started at http://127.21.236.190:35323/ using document root <none> and password file <none>
I20260811 02:47:06.186626 30337 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:06.186695 30337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:06.186921 30337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:06.189193 30337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "febb6f7ec84849189deb2b85b4f6775d"
format_stamp: "Formatted at 2026-08-11 02:47:06 on dist-test-slave-n326"
server_key: "2f39ce14e1fe8d49db88261fee851fd0"
server_key_iv: "0900c8f53f608fcec1182dafde97563c"
server_key_version: "encryptionkey@0"
I20260811 02:47:06.189771 30337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "febb6f7ec84849189deb2b85b4f6775d"
format_stamp: "Formatted at 2026-08-11 02:47:06 on dist-test-slave-n326"
server_key: "2f39ce14e1fe8d49db88261fee851fd0"
server_key_iv: "0900c8f53f608fcec1182dafde97563c"
server_key_version: "encryptionkey@0"
I20260811 02:47:06.193674 30337 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:47:06.196161 30364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:06.197342 30337 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:47:06.197482 30337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "febb6f7ec84849189deb2b85b4f6775d"
format_stamp: "Formatted at 2026-08-11 02:47:06 on dist-test-slave-n326"
server_key: "2f39ce14e1fe8d49db88261fee851fd0"
server_key_iv: "0900c8f53f608fcec1182dafde97563c"
server_key_version: "encryptionkey@0"
I20260811 02:47:06.197629 30337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:06.202348 30337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:06.202962 30337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:06.203179 30337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:06.210970 30337 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:35885
I20260811 02:47:06.210978 30448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:35885 every 8 connection(s)
I20260811 02:47:06.211988 30337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:06.212374 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 30337
I20260811 02:47:06.212536 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:47:06.212817 22450 external_mini_cluster.cc:1428] Setting key 0513e43ecbd4a763f1a20c35c4af35fa
I20260811 02:47:06.215421 30449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:06.219615 30452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:06.214240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48694 (local address 127.21.236.190:35885)
0811 02:47:06.215686 (+ 1446us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:06.215702 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:06.215742 (+ 40us) server_negotiation.cc:407] Connection header received
0811 02:47:06.216465 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:06.216484 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:06.216852 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:06.217234 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:06.218160 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:06.218165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:06.218178 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:06.218189 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:06.218255 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:06.218272 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:06.218450 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:06.218531 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 02:47:06.218796 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":946,"thread_start_us":739,"threads_started":1}
I20260811 02:47:06.221508 30449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d: Bootstrap starting.
I20260811 02:47:06.224005 30449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:06.224794 30449 log.cc:824] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:06.226830 30449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d: No bootstrap required, opened a new log
I20260811 02:47:06.231215 30449 raft_consensus.cc:348] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "febb6f7ec84849189deb2b85b4f6775d" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 35885 } }
I20260811 02:47:06.231483 30449 raft_consensus.cc:374] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:06.231568 30449 raft_consensus.cc:729] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: febb6f7ec84849189deb2b85b4f6775d, State: Initialized, Role: FOLLOWER
I20260811 02:47:06.232054 30449 consensus_queue.cc:260] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "febb6f7ec84849189deb2b85b4f6775d" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 35885 } }
I20260811 02:47:06.232210 30449 raft_consensus.cc:388] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:06.232300 30449 raft_consensus.cc:482] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:06.232403 30449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:06.233379 30449 raft_consensus.cc:504] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "febb6f7ec84849189deb2b85b4f6775d" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 35885 } }
I20260811 02:47:06.233737 30449 leader_election.cc:302] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: febb6f7ec84849189deb2b85b4f6775d; no voters:
I20260811 02:47:06.234127 30449 leader_election.cc:288] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:47:06.234229 30454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:06.234495 30454 raft_consensus.cc:686] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [term 1 LEADER]: Becoming Leader. State: Replica: febb6f7ec84849189deb2b85b4f6775d, State: Running, Role: LEADER
I20260811 02:47:06.234833 30454 consensus_queue.cc:237] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "febb6f7ec84849189deb2b85b4f6775d" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 35885 } }
I20260811 02:47:06.235630 30449 sys_catalog.cc:564] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:06.236945 30456 sys_catalog.cc:455] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "febb6f7ec84849189deb2b85b4f6775d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "febb6f7ec84849189deb2b85b4f6775d" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 35885 } } }
I20260811 02:47:06.237094 30456 sys_catalog.cc:458] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:06.236930 30457 sys_catalog.cc:455] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [sys.catalog]: SysCatalogTable state changed. Reason: New leader febb6f7ec84849189deb2b85b4f6775d. Latest consensus state: current_term: 1 leader_uuid: "febb6f7ec84849189deb2b85b4f6775d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "febb6f7ec84849189deb2b85b4f6775d" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 35885 } } }
I20260811 02:47:06.237278 30457 sys_catalog.cc:458] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:06.238195 30473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:06.240834 30473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:06.246016 30473 catalog_manager.cc:1347] Generated new cluster ID: 9dca85e8600147d5adffcadf3731bac3
I20260811 02:47:06.246088 30473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:06.277503 30473 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:47:06.277879 30473 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:06.287997 30473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d: Generated new TSK 0
I20260811 02:47:06.288888 30473 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:47:06.295454 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:35885
--builtin_ntp_servers=127.21.236.148:45579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 02:47:06.415592 30492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:06.415866 30492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:06.415907 30492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:06.420003 30492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:06.420073 30492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:06.420130 30492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:47:06.425067 30492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:35885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:06.426290 30492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:06.427850 30492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:06.438513 30498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:06.438611 30503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:06.438513 30499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:06.438925 30492 server_base.cc:1034] running on GCE node
I20260811 02:47:06.439419 30492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:06.440081 30492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:06.441256 30492 hybrid_clock.cc:648] HybridClock initialized: now 1786416426441228 us; error 39 us; skew 500 ppm
I20260811 02:47:06.443456 30492 webserver.cc:460] Webserver started at http://127.21.236.129:40563/ using document root <none> and password file <none>
I20260811 02:47:06.444073 30492 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:06.444145 30492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:06.444329 30492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:06.446579 30492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "240416a4b2054cff96362d3d2d3756e3"
format_stamp: "Formatted at 2026-08-11 02:47:06 on dist-test-slave-n326"
server_key: "d59db7549257f6dbe6e951bea30e471f"
server_key_iv: "d6c2f0caf63cf006cf748b3408f58327"
server_key_version: "encryptionkey@0"
I20260811 02:47:06.447223 30492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "240416a4b2054cff96362d3d2d3756e3"
format_stamp: "Formatted at 2026-08-11 02:47:06 on dist-test-slave-n326"
server_key: "d59db7549257f6dbe6e951bea30e471f"
server_key_iv: "d6c2f0caf63cf006cf748b3408f58327"
server_key_version: "encryptionkey@0"
I20260811 02:47:06.451395 30492 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:47:06.454306 30511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:06.455793 30492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:47:06.455977 30492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "240416a4b2054cff96362d3d2d3756e3"
format_stamp: "Formatted at 2026-08-11 02:47:06 on dist-test-slave-n326"
server_key: "d59db7549257f6dbe6e951bea30e471f"
server_key_iv: "d6c2f0caf63cf006cf748b3408f58327"
server_key_version: "encryptionkey@0"
I20260811 02:47:06.456086 30492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:06.460841 30492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:06.461488 30492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:06.461680 30492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:06.462265 30492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:06.463389 30492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:06.463441 30492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:06.463480 30492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:06.463538 30492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:06.472679 30492 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:42899
I20260811 02:47:06.472720 30685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:42899 every 8 connection(s)
I20260811 02:47:06.473645 30492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:47:06.479936 30452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:06.475540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:37837 (local address 127.21.236.190:35885)
0811 02:47:06.475696 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:06.475700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:06.476735 (+ 1035us) server_negotiation.cc:407] Connection header received
0811 02:47:06.478101 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:06.478104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:06.478150 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:06.478234 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:06.479377 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:06.479381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:06.479384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:06.479399 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:06.479426 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:06.479437 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:06.479539 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:06.479707 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 02:47:06.479757 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:47:06.480866 30692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:06.475892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:35885 (local address 127.21.236.129:37837)
0811 02:47:06.476579 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:06.476620 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:06.477865 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:06.478404 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:06.478411 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:06.478495 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:06.479080 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:47:06.479082 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:06.479127 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:47:06.479202 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:06.479534 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:06.479545 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:06.479553 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 02:47:06.479753 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 02:47:06.480026 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":122,"threads_started":1}
I20260811 02:47:06.482388 30687 heartbeater.cc:347] Connected to a master server at 127.21.236.190:35885
I20260811 02:47:06.482645 30687 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:06.483153 30687 heartbeater.cc:511] Master 127.21.236.190:35885 requested a full tablet report, sending...
I20260811 02:47:06.483294 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 30492
I20260811 02:47:06.483450 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:47:06.483770 22450 external_mini_cluster.cc:1428] Setting key ffb79d7eb87ddcf1ccc37b9489246d35
I20260811 02:47:06.484992 30383 ts_manager.cc:194] Registered new tserver with Master: 240416a4b2054cff96362d3d2d3756e3 (127.21.236.129:42899)
I20260811 02:47:06.485986 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:35885
--builtin_ntp_servers=127.21.236.148:45579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 02:47:06.602031 30694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:06.602358 30694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:06.602466 30694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:06.606949 30694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:06.607036 30694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:06.607141 30694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:47:06.611976 30694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:35885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:06.613085 30694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:06.614503 30694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:06.624368 30700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:06.624357 30704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:06.624357 30707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:06.624773 30694 server_base.cc:1034] running on GCE node
I20260811 02:47:06.625243 30694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:06.625905 30694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:06.627163 30694 hybrid_clock.cc:648] HybridClock initialized: now 1786416426627137 us; error 35 us; skew 500 ppm
I20260811 02:47:06.629343 30694 webserver.cc:460] Webserver started at http://127.21.236.130:33839/ using document root <none> and password file <none>
I20260811 02:47:06.630019 30694 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:06.630126 30694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:06.630389 30694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:06.632800 30694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "319a8ea8b18d4706b500bb4b8840bb9b"
format_stamp: "Formatted at 2026-08-11 02:47:06 on dist-test-slave-n326"
server_key: "4c6a878574895a7c76d9a932d43e4910"
server_key_iv: "810db340a66670169fda402efe54e44e"
server_key_version: "encryptionkey@0"
I20260811 02:47:06.633476 30694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "319a8ea8b18d4706b500bb4b8840bb9b"
format_stamp: "Formatted at 2026-08-11 02:47:06 on dist-test-slave-n326"
server_key: "4c6a878574895a7c76d9a932d43e4910"
server_key_iv: "810db340a66670169fda402efe54e44e"
server_key_version: "encryptionkey@0"
I20260811 02:47:06.638033 30694 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:47:06.640983 30716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:06.642478 30694 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:47:06.642639 30694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "319a8ea8b18d4706b500bb4b8840bb9b"
format_stamp: "Formatted at 2026-08-11 02:47:06 on dist-test-slave-n326"
server_key: "4c6a878574895a7c76d9a932d43e4910"
server_key_iv: "810db340a66670169fda402efe54e44e"
server_key_version: "encryptionkey@0"
I20260811 02:47:06.642769 30694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:06.647676 30694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:06.648319 30694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:06.648517 30694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:06.649101 30694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:06.650068 30694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:06.650143 30694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:06.650218 30694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:06.650264 30694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:06.660115 30694 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:43277
I20260811 02:47:06.660254 30928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:43277 every 8 connection(s)
I20260811 02:47:06.661128 30694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:47:06.663753 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 30694
I20260811 02:47:06.663882 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:47:06.664249 22450 external_mini_cluster.cc:1428] Setting key 6640adaf5ea370565cf38318fe14633a
I20260811 02:47:06.666313 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:35885
--builtin_ntp_servers=127.21.236.148:45579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 02:47:06.668404 30452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:06.663202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:40209 (local address 127.21.236.190:35885)
0811 02:47:06.663362 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:06.663366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:06.664790 (+ 1424us) server_negotiation.cc:407] Connection header received
0811 02:47:06.666388 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:06.666392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:06.666435 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:06.666542 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:06.667878 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:06.667882 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:06.667897 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:06.667914 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:06.667936 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:06.667944 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:06.668067 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:06.668222 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 02:47:06.668270 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:47:06.669342 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:06.663752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:35885 (local address 127.21.236.130:40209)
0811 02:47:06.664575 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:06.664649 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:06.666166 (+ 1517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:06.666727 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:06.666735 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:06.666832 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:06.667593 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:47:06.667595 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:06.667638 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:47:06.667718 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:06.668053 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:06.668063 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:06.668069 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:47:06.668262 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 02:47:06.668535 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":166,"threads_started":1}
I20260811 02:47:06.670701 30930 heartbeater.cc:347] Connected to a master server at 127.21.236.190:35885
I20260811 02:47:06.671005 30930 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:06.671581 30930 heartbeater.cc:511] Master 127.21.236.190:35885 requested a full tablet report, sending...
I20260811 02:47:06.672884 30383 ts_manager.cc:194] Registered new tserver with Master: 319a8ea8b18d4706b500bb4b8840bb9b (127.21.236.130:43277)
W20260811 02:47:06.787328 30936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:06.787652 30936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:06.787719 30936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:06.792178 30936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:06.792240 30936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:06.792299 30936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:47:06.797127 30936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:35885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:06.798211 30936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:06.799664 30936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:06.809718 30945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:06.809729 30942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:06.809710 30943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:06.810156 30936 server_base.cc:1034] running on GCE node
I20260811 02:47:06.810611 30936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:06.811337 30936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:06.812530 30936 hybrid_clock.cc:648] HybridClock initialized: now 1786416426812495 us; error 49 us; skew 500 ppm
I20260811 02:47:06.814465 30936 webserver.cc:460] Webserver started at http://127.21.236.131:44273/ using document root <none> and password file <none>
I20260811 02:47:06.815130 30936 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:06.815201 30936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:06.815439 30936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:06.817540 30936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "947ba74cd4de438c9c8326b315695764"
format_stamp: "Formatted at 2026-08-11 02:47:06 on dist-test-slave-n326"
server_key: "5caa49374a44252fb49edc4de60c31ad"
server_key_iv: "5383efb77d2a5aaa760e03a2704e72a2"
server_key_version: "encryptionkey@0"
I20260811 02:47:06.818120 30936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "947ba74cd4de438c9c8326b315695764"
format_stamp: "Formatted at 2026-08-11 02:47:06 on dist-test-slave-n326"
server_key: "5caa49374a44252fb49edc4de60c31ad"
server_key_iv: "5383efb77d2a5aaa760e03a2704e72a2"
server_key_version: "encryptionkey@0"
I20260811 02:47:06.821945 30936 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:47:06.824855 30955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:06.826349 30936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 02:47:06.826547 30936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "947ba74cd4de438c9c8326b315695764"
format_stamp: "Formatted at 2026-08-11 02:47:06 on dist-test-slave-n326"
server_key: "5caa49374a44252fb49edc4de60c31ad"
server_key_iv: "5383efb77d2a5aaa760e03a2704e72a2"
server_key_version: "encryptionkey@0"
I20260811 02:47:06.826679 30936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:06.831847 30936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:06.832525 30936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:06.832731 30936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:06.833338 30936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:06.834373 30936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:06.834455 30936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:06.834537 30936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:06.834594 30936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:06.844605 30936 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:33521
I20260811 02:47:06.844635 31143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:33521 every 8 connection(s)
I20260811 02:47:06.845616 30936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:47:06.852742 30452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:06.847936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:53759 (local address 127.21.236.190:35885)
0811 02:47:06.848081 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:06.848084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:06.849205 (+ 1121us) server_negotiation.cc:407] Connection header received
0811 02:47:06.850839 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:06.850843 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:06.850894 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:06.850983 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:06.852198 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:06.852202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:06.852205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:06.852225 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:06.852247 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:06.852257 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:06.852357 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:06.852538 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 02:47:06.852589 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:47:06.853696 31150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:06.848253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:35885 (local address 127.21.236.131:53759)
0811 02:47:06.849014 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:06.849058 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:06.850599 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:06.851190 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:06.851198 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:06.851295 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:06.851860 (+ 565us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:47:06.851862 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:06.851931 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:47:06.852007 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:06.852351 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:06.852361 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:06.852368 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 02:47:06.852587 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 02:47:06.852851 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":137,"threads_started":1}
I20260811 02:47:06.854856 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 30936
I20260811 02:47:06.854923 31145 heartbeater.cc:347] Connected to a master server at 127.21.236.190:35885
I20260811 02:47:06.855003 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:47:06.855212 31145 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:06.855413 22450 external_mini_cluster.cc:1428] Setting key 7680631d606e0f059eb4f667cc261b87
I20260811 02:47:06.855688 31145 heartbeater.cc:511] Master 127.21.236.190:35885 requested a full tablet report, sending...
I20260811 02:47:06.856979 30383 ts_manager.cc:194] Registered new tserver with Master: 947ba74cd4de438c9c8326b315695764 (127.21.236.131:33521)
I20260811 02:47:06.857858 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:47:06.861680 30452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:06.859894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57452 (local address 127.21.236.190:35885)
0811 02:47:06.860026 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:06.860030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:06.860289 (+ 259us) server_negotiation.cc:407] Connection header received
0811 02:47:06.860453 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:06.860456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:06.860492 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:06.860583 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:06.861271 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:06.861275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:06.861277 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:06.861288 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:06.861309 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:06.861318 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:06.861438 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:06.861507 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 02:47:06.861550 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:47:06.864758 30390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:47:06.867865 30390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:47:06.880401 31168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:06.876912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:33521 (local address 127.0.0.1:33806)
0811 02:47:06.877972 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:06.877986 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:06.878066 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:06.878836 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:06.878843 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:06.878918 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:06.879575 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:47:06.879579 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:06.879604 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:47:06.879680 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:06.880035 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:06.880043 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:06.880049 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:47:06.880173 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 02:47:06.880245 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":997,"thread_start_us":120,"threads_started":1}
I20260811 02:47:06.880474 31169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:06.877745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33806 (local address 127.21.236.131:33521)
0811 02:47:06.878347 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:06.878363 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:06.878422 (+ 59us) server_negotiation.cc:407] Connection header received
0811 02:47:06.878481 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:06.878488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:06.878665 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:06.879226 (+ 561us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:06.879857 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:06.879863 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:06.879875 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:06.879887 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:06.879926 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:06.879958 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:06.880068 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:06.880201 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 02:47:06.880281 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":164,"threads_started":1}
I20260811 02:47:06.880908 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:06.877089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40938 (local address 127.21.236.129:42899)
0811 02:47:06.877439 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:06.877452 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:06.877910 (+ 458us) server_negotiation.cc:407] Connection header received
0811 02:47:06.877993 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:06.878003 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:06.878219 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:06.878368 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:06.879991 (+ 1623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:06.879997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:06.880010 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:06.880024 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:06.880071 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:06.880106 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:06.880207 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:06.880662 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 02:47:06.880733 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":138,"threads_started":1}
I20260811 02:47:06.880448 31164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:06.876890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42899 (local address 127.0.0.1:40938)
0811 02:47:06.877628 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:06.877680 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:06.877793 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:06.879047 (+ 1254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:06.879049 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:06.879117 (+ 68us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:06.879823 (+ 706us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:47:06.879826 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:06.879838 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:47:06.879861 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:06.880246 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:06.880248 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:06.880251 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 02:47:06.880315 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 02:47:06.880341 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":101,"threads_started":1}
I20260811 02:47:06.881258 31165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:06.877651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:43277 (local address 127.0.0.1:55338)
0811 02:47:06.878321 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:06.878335 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:06.878412 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:06.879203 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:06.879206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:06.879221 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:06.879869 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:47:06.879871 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:06.879894 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:47:06.879922 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:06.880450 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:06.880453 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:06.880456 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 02:47:06.880728 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 02:47:06.880769 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":101,"threads_started":1}
I20260811 02:47:06.881608 31170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:06.877850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55338 (local address 127.21.236.130:43277)
0811 02:47:06.878654 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:06.878665 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:06.878702 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:47:06.878756 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:06.878763 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:06.879017 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:06.879418 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:06.880257 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:06.880262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:06.880270 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:06.880280 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:06.880316 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:06.880338 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:06.880893 (+ 555us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:06.880941 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 02:47:06.881446 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":147,"threads_started":1}
I20260811 02:47:06.883579 31034 tablet_service.cc:1467] Processing CreateTablet for tablet 52f5a4db47f3476faa77cdf6419924c3 (DEFAULT_TABLE table=test-table [id=0522a1cf0d2241e0bf4caee0250892ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:06.884687 31034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52f5a4db47f3476faa77cdf6419924c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:06.884280 30799 tablet_service.cc:1467] Processing CreateTablet for tablet 52f5a4db47f3476faa77cdf6419924c3 (DEFAULT_TABLE table=test-table [id=0522a1cf0d2241e0bf4caee0250892ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:06.884282 30583 tablet_service.cc:1467] Processing CreateTablet for tablet 52f5a4db47f3476faa77cdf6419924c3 (DEFAULT_TABLE table=test-table [id=0522a1cf0d2241e0bf4caee0250892ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:06.885352 30799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52f5a4db47f3476faa77cdf6419924c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:06.885352 30583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52f5a4db47f3476faa77cdf6419924c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:06.891218 31171 tablet_bootstrap.cc:492] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764: Bootstrap starting.
I20260811 02:47:06.893054 31173 tablet_bootstrap.cc:492] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3: Bootstrap starting.
I20260811 02:47:06.894238 31171 tablet_bootstrap.cc:654] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:06.895232 31171 log.cc:824] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:06.895465 31173 tablet_bootstrap.cc:654] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:06.895493 31172 tablet_bootstrap.cc:492] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b: Bootstrap starting.
I20260811 02:47:06.896304 31173 log.cc:824] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:06.897631 31171 tablet_bootstrap.cc:492] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764: No bootstrap required, opened a new log
I20260811 02:47:06.897620 31172 tablet_bootstrap.cc:654] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:06.897840 31171 ts_tablet_manager.cc:1397] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260811 02:47:06.898090 31173 tablet_bootstrap.cc:492] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3: No bootstrap required, opened a new log
I20260811 02:47:06.898290 31173 ts_tablet_manager.cc:1397] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:47:06.898458 31172 log.cc:824] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:06.900471 31172 tablet_bootstrap.cc:492] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b: No bootstrap required, opened a new log
I20260811 02:47:06.900653 31172 ts_tablet_manager.cc:1397] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:47:06.902495 31171 raft_consensus.cc:348] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "240416a4b2054cff96362d3d2d3756e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42899 } } peers { permanent_uuid: "947ba74cd4de438c9c8326b315695764" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33521 } } peers { permanent_uuid: "319a8ea8b18d4706b500bb4b8840bb9b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43277 } }
I20260811 02:47:06.902781 31171 raft_consensus.cc:374] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:06.902624 31173 raft_consensus.cc:348] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "240416a4b2054cff96362d3d2d3756e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42899 } } peers { permanent_uuid: "947ba74cd4de438c9c8326b315695764" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33521 } } peers { permanent_uuid: "319a8ea8b18d4706b500bb4b8840bb9b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43277 } }
I20260811 02:47:06.902842 31171 raft_consensus.cc:729] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947ba74cd4de438c9c8326b315695764, State: Initialized, Role: FOLLOWER
I20260811 02:47:06.902858 31173 raft_consensus.cc:374] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:06.902906 31173 raft_consensus.cc:729] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 240416a4b2054cff96362d3d2d3756e3, State: Initialized, Role: FOLLOWER
I20260811 02:47:06.903360 31173 consensus_queue.cc:260] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "240416a4b2054cff96362d3d2d3756e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42899 } } peers { permanent_uuid: "947ba74cd4de438c9c8326b315695764" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33521 } } peers { permanent_uuid: "319a8ea8b18d4706b500bb4b8840bb9b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43277 } }
I20260811 02:47:06.903326 31171 consensus_queue.cc:260] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "240416a4b2054cff96362d3d2d3756e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42899 } } peers { permanent_uuid: "947ba74cd4de438c9c8326b315695764" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33521 } } peers { permanent_uuid: "319a8ea8b18d4706b500bb4b8840bb9b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43277 } }
I20260811 02:47:06.904066 30687 heartbeater.cc:503] Master 127.21.236.190:35885 was elected leader, sending a full tablet report...
I20260811 02:47:06.904371 31173 ts_tablet_manager.cc:1428] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:47:06.904397 31171 ts_tablet_manager.cc:1428] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260811 02:47:06.904735 31145 heartbeater.cc:503] Master 127.21.236.190:35885 was elected leader, sending a full tablet report...
I20260811 02:47:06.905671 31172 raft_consensus.cc:348] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "240416a4b2054cff96362d3d2d3756e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42899 } } peers { permanent_uuid: "947ba74cd4de438c9c8326b315695764" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33521 } } peers { permanent_uuid: "319a8ea8b18d4706b500bb4b8840bb9b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43277 } }
I20260811 02:47:06.905949 31172 raft_consensus.cc:374] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:06.906003 31172 raft_consensus.cc:729] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 319a8ea8b18d4706b500bb4b8840bb9b, State: Initialized, Role: FOLLOWER
I20260811 02:47:06.906391 31172 consensus_queue.cc:260] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "240416a4b2054cff96362d3d2d3756e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42899 } } peers { permanent_uuid: "947ba74cd4de438c9c8326b315695764" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33521 } } peers { permanent_uuid: "319a8ea8b18d4706b500bb4b8840bb9b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43277 } }
I20260811 02:47:06.907249 30930 heartbeater.cc:503] Master 127.21.236.190:35885 was elected leader, sending a full tablet report...
I20260811 02:47:06.907418 31172 ts_tablet_manager.cc:1428] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260811 02:47:06.912103 30931 tablet.cc:2369] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:47:06.974942 30691 tablet.cc:2369] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:47:07.096947 31147 tablet.cc:2369] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:47:07.143380 31178 raft_consensus.cc:482] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:47:07.143833 31178 raft_consensus.cc:504] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "240416a4b2054cff96362d3d2d3756e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42899 } } peers { permanent_uuid: "947ba74cd4de438c9c8326b315695764" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33521 } } peers { permanent_uuid: "319a8ea8b18d4706b500bb4b8840bb9b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43277 } }
I20260811 02:47:07.145359 31178 leader_election.cc:288] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 947ba74cd4de438c9c8326b315695764 (127.21.236.131:33521), 319a8ea8b18d4706b500bb4b8840bb9b (127.21.236.130:43277)
I20260811 02:47:07.147846 31169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:07.145765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:60559 (local address 127.21.236.131:33521)
0811 02:47:07.145943 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:07.145948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:07.146187 (+ 239us) server_negotiation.cc:407] Connection header received
0811 02:47:07.146336 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:07.146339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:07.146383 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:07.146485 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:07.147406 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:07.147410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:07.147413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:07.147427 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:07.147455 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:07.147464 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:07.147547 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:07.147636 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 02:47:07.147701 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 02:47:07.147881 31182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:07.145664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:33521 (local address 127.21.236.129:60559)
0811 02:47:07.146083 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:07.146095 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:07.146220 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:07.146511 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:07.146514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:07.146531 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:07.147159 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:47:07.147161 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:07.147174 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:47:07.147208 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:07.147554 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:07.147557 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:07.147560 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 02:47:07.147650 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 02:47:07.147694 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":107,"threads_started":1}
I20260811 02:47:07.147946 31170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:07.145816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:44067 (local address 127.21.236.130:43277)
0811 02:47:07.145993 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:07.145998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:07.146250 (+ 252us) server_negotiation.cc:407] Connection header received
0811 02:47:07.146491 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:07.146494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:07.146533 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:07.146633 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:07.147534 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:07.147540 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:07.147543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:07.147555 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:07.147580 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:07.147594 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:07.147687 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:07.147782 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 02:47:07.147817 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 02:47:07.148203 31183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:07.145656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:43277 (local address 127.21.236.129:44067)
0811 02:47:07.146012 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:07.146031 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:07.146295 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:07.146652 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:07.146656 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:07.146673 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:07.147311 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:47:07.147315 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:07.147328 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:47:07.147352 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:07.147695 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:07.147698 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:07.147702 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 02:47:07.147785 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 02:47:07.147818 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":82,"threads_started":1}
I20260811 02:47:07.148690 31063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52f5a4db47f3476faa77cdf6419924c3" candidate_uuid: "240416a4b2054cff96362d3d2d3756e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947ba74cd4de438c9c8326b315695764" is_pre_election: true
I20260811 02:47:07.148897 30845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52f5a4db47f3476faa77cdf6419924c3" candidate_uuid: "240416a4b2054cff96362d3d2d3756e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "319a8ea8b18d4706b500bb4b8840bb9b" is_pre_election: true
I20260811 02:47:07.149025 31063 raft_consensus.cc:2454] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 240416a4b2054cff96362d3d2d3756e3 in term 0.
I20260811 02:47:07.149158 30845 raft_consensus.cc:2454] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 240416a4b2054cff96362d3d2d3756e3 in term 0.
I20260811 02:47:07.149566 30514 leader_election.cc:302] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 240416a4b2054cff96362d3d2d3756e3, 947ba74cd4de438c9c8326b315695764; no voters:
I20260811 02:47:07.149806 31178 raft_consensus.cc:2781] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:47:07.149899 31178 raft_consensus.cc:482] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:47:07.149946 31178 raft_consensus.cc:3037] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:07.151016 31178 raft_consensus.cc:504] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "240416a4b2054cff96362d3d2d3756e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42899 } } peers { permanent_uuid: "947ba74cd4de438c9c8326b315695764" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33521 } } peers { permanent_uuid: "319a8ea8b18d4706b500bb4b8840bb9b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43277 } }
I20260811 02:47:07.151489 31178 leader_election.cc:288] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [CANDIDATE]: Term 1 election: Requested vote from peers 947ba74cd4de438c9c8326b315695764 (127.21.236.131:33521), 319a8ea8b18d4706b500bb4b8840bb9b (127.21.236.130:43277)
I20260811 02:47:07.151895 30845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52f5a4db47f3476faa77cdf6419924c3" candidate_uuid: "240416a4b2054cff96362d3d2d3756e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "319a8ea8b18d4706b500bb4b8840bb9b"
I20260811 02:47:07.151904 31063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52f5a4db47f3476faa77cdf6419924c3" candidate_uuid: "240416a4b2054cff96362d3d2d3756e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947ba74cd4de438c9c8326b315695764"
I20260811 02:47:07.152015 30845 raft_consensus.cc:3037] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:07.152069 31063 raft_consensus.cc:3037] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:07.153132 30845 raft_consensus.cc:2454] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 240416a4b2054cff96362d3d2d3756e3 in term 1.
I20260811 02:47:07.153147 31063 raft_consensus.cc:2454] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 240416a4b2054cff96362d3d2d3756e3 in term 1.
I20260811 02:47:07.153466 30512 leader_election.cc:302] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 240416a4b2054cff96362d3d2d3756e3, 319a8ea8b18d4706b500bb4b8840bb9b; no voters:
I20260811 02:47:07.153652 31178 raft_consensus.cc:2781] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:07.153844 31178 raft_consensus.cc:686] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [term 1 LEADER]: Becoming Leader. State: Replica: 240416a4b2054cff96362d3d2d3756e3, State: Running, Role: LEADER
I20260811 02:47:07.154188 31178 consensus_queue.cc:237] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "240416a4b2054cff96362d3d2d3756e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42899 } } peers { permanent_uuid: "947ba74cd4de438c9c8326b315695764" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33521 } } peers { permanent_uuid: "319a8ea8b18d4706b500bb4b8840bb9b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43277 } }
I20260811 02:47:07.157456 30389 catalog_manager.cc:5495] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 240416a4b2054cff96362d3d2d3756e3 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "240416a4b2054cff96362d3d2d3756e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "240416a4b2054cff96362d3d2d3756e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947ba74cd4de438c9c8326b315695764" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "319a8ea8b18d4706b500bb4b8840bb9b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43277 } health_report { overall_health: UNKNOWN } } }
I20260811 02:47:07.176939 31166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:07.175306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40948 (local address 127.21.236.129:42899)
0811 02:47:07.175466 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:07.175470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:07.175485 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:47:07.175668 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:07.175671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:07.175736 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:07.175815 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:07.176549 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:07.176552 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:07.176554 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:07.176562 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:07.176578 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:07.176585 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:07.176658 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:07.176799 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 02:47:07.176834 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 02:47:07.183319 30845 raft_consensus.cc:1261] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b [term 1 FOLLOWER]: Refusing update from remote peer 240416a4b2054cff96362d3d2d3756e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:07.183319 31063 raft_consensus.cc:1261] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764 [term 1 FOLLOWER]: Refusing update from remote peer 240416a4b2054cff96362d3d2d3756e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:07.184114 31184 consensus_queue.cc:1035] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947ba74cd4de438c9c8326b315695764" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:07.184320 31178 consensus_queue.cc:1035] T 52f5a4db47f3476faa77cdf6419924c3 P 240416a4b2054cff96362d3d2d3756e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "319a8ea8b18d4706b500bb4b8840bb9b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:07.201639 30383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:07.201937 30383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:07.204754 30383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P febb6f7ec84849189deb2b85b4f6775d: Sending DeleteTablet for 3 replicas of tablet 52f5a4db47f3476faa77cdf6419924c3
I20260811 02:47:07.205644 31034 tablet_service.cc:1514] Processing DeleteTablet for tablet 52f5a4db47f3476faa77cdf6419924c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:07 UTC) from {username='slave'} at 127.0.0.1:33806
I20260811 02:47:07.205696 30583 tablet_service.cc:1514] Processing DeleteTablet for tablet 52f5a4db47f3476faa77cdf6419924c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:07 UTC) from {username='slave'} at 127.0.0.1:40938
I20260811 02:47:07.206236 30799 tablet_service.cc:1514] Processing DeleteTablet for tablet 52f5a4db47f3476faa77cdf6419924c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:07 UTC) from {username='slave'} at 127.0.0.1:55338
I20260811 02:47:07.206845 31214 tablet_replica.cc:330] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b: stopping tablet replica
I20260811 02:47:07.207101 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 30492
I20260811 02:47:07.207099 31213 tablet_replica.cc:330] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764: stopping tablet replica
I20260811 02:47:07.207377 31214 raft_consensus.cc:2229] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:07.207523 31213 raft_consensus.cc:2229] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:07.207854 31214 raft_consensus.cc:2258] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:07.207894 31213 raft_consensus.cc:2258] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:07.212489 31213 ts_tablet_manager.cc:1905] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:07.215317 31214 ts_tablet_manager.cc:1905] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:07.216682 31213 ts_tablet_manager.cc:1918] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:07.216825 31213 log.cc:1196] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/52f5a4db47f3476faa77cdf6419924c3
I20260811 02:47:07.217204 31213 ts_tablet_manager.cc:1939] T 52f5a4db47f3476faa77cdf6419924c3 P 947ba74cd4de438c9c8326b315695764: Deleting consensus metadata
I20260811 02:47:07.218502 31214 ts_tablet_manager.cc:1918] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:07.218721 31214 log.cc:1196] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/52f5a4db47f3476faa77cdf6419924c3
I20260811 02:47:07.218858 30370 catalog_manager.cc:4841] TS 947ba74cd4de438c9c8326b315695764 (127.21.236.131:33521): tablet 52f5a4db47f3476faa77cdf6419924c3 (table test-table [id=0522a1cf0d2241e0bf4caee0250892ca]) successfully deleted
I20260811 02:47:07.219202 31214 ts_tablet_manager.cc:1939] T 52f5a4db47f3476faa77cdf6419924c3 P 319a8ea8b18d4706b500bb4b8840bb9b: Deleting consensus metadata
I20260811 02:47:07.220535 30367 catalog_manager.cc:4841] TS 319a8ea8b18d4706b500bb4b8840bb9b (127.21.236.130:43277): tablet 52f5a4db47f3476faa77cdf6419924c3 (table test-table [id=0522a1cf0d2241e0bf4caee0250892ca]) successfully deleted
W20260811 02:47:07.224326 30367 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:42899 (error 108)
I20260811 02:47:07.224468 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 30694
I20260811 02:47:07.236635 31165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:07.236217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42899 (local address 127.0.0.1:40954)
0811 02:47:07.236434 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:07.236515 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:42899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260811 02:47:07.236886 30367 catalog_manager.cc:4584] TS 240416a4b2054cff96362d3d2d3756e3 (127.21.236.129:42899): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 52f5a4db47f3476faa77cdf6419924c3: Network error: Client connection negotiation failed: client connection to 127.21.236.129:42899: connect: Connection refused (error 111)
I20260811 02:47:07.243031 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 30936
I20260811 02:47:07.263757 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 30337
2026-08-11T02:47:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1256 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 02:47:07.293731 22450 mini_oidc.cc:152] Starting JWKS server
I20260811 02:47:07.295730 22450 webserver.cc:460] Webserver started at https://127.0.0.1:43681/ using document root <none> and password file <none>
I20260811 02:47:07.295905 22450 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 02:47:07.296166 22450 webserver.cc:460] Webserver started at http://127.0.0.1:35521/ using document root <none> and password file <none>
2026-08-11T02:47:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:07Z Disabled control of system clock
I20260811 02:47:07.307785 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:39711
--webserver_interface=127.21.236.190
--webserver_port=0
--builtin_ntp_servers=127.21.236.148:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:39711
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43681/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/testchainca.cert with env {}
W20260811 02:47:07.430074 31223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:07.430399 31223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:07.430516 31223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:07.434715 31223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:07.434779 31223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:07.434796 31223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:07.434813 31223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:07.434828 31223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:47:07.434890 31223 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 02:47:07.434916 31223 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43681/jwks.json
I20260811 02:47:07.440418 31223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:39711
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:39711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43681/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:07.441598 31223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:07.442687 31223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:07.451645 31230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:07.451640 31229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:07.451694 31223 server_base.cc:1034] running on GCE node
W20260811 02:47:07.451640 31234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:07.452517 31223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:07.453617 31223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:07.454803 31223 hybrid_clock.cc:648] HybridClock initialized: now 1786416427454781 us; error 39 us; skew 500 ppm
I20260811 02:47:07.456800 31223 webserver.cc:460] Webserver started at http://127.21.236.190:44783/ using document root <none> and password file <none>
I20260811 02:47:07.457363 31223 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:07.457427 31223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:07.457603 31223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:07.459743 31223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "d0b920730b42439b8676583a45104e56"
format_stamp: "Formatted at 2026-08-11 02:47:07 on dist-test-slave-n326"
server_key: "95b06363e5cea762a3135d5f25e8caa3"
server_key_iv: "8c3899d4ee13dcb1f5b423a1c6c312ed"
server_key_version: "encryptionkey@0"
I20260811 02:47:07.460278 31223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "d0b920730b42439b8676583a45104e56"
format_stamp: "Formatted at 2026-08-11 02:47:07 on dist-test-slave-n326"
server_key: "95b06363e5cea762a3135d5f25e8caa3"
server_key_iv: "8c3899d4ee13dcb1f5b423a1c6c312ed"
server_key_version: "encryptionkey@0"
I20260811 02:47:07.464205 31223 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:47:07.466972 31244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:07.468524 31223 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:47:07.468647 31223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "d0b920730b42439b8676583a45104e56"
format_stamp: "Formatted at 2026-08-11 02:47:07 on dist-test-slave-n326"
server_key: "95b06363e5cea762a3135d5f25e8caa3"
server_key_iv: "8c3899d4ee13dcb1f5b423a1c6c312ed"
server_key_version: "encryptionkey@0"
I20260811 02:47:07.468807 31223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:07.481477 31223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:07.482226 31223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:07.482451 31223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:07.489881 31223 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:39711
I20260811 02:47:07.489900 31325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:39711 every 8 connection(s)
I20260811 02:47:07.491075 31223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:07.494052 31326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:07.495203 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 31223
I20260811 02:47:07.495344 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:47:07.495642 22450 external_mini_cluster.cc:1428] Setting key bf9a4949cfe48d48893977750fc2e089
I20260811 02:47:07.501066 31326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56: Bootstrap starting.
I20260811 02:47:07.503377 31326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:07.504328 31326 log.cc:824] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:07.505867 31334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:07.497386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45972 (local address 127.21.236.190:39711)
0811 02:47:07.498074 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:07.498102 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:07.498156 (+ 54us) server_negotiation.cc:407] Connection header received
0811 02:47:07.499180 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:07.499200 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:07.499600 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:07.499880 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:07.500231 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:07.501157 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:07.502069 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:07.502562 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:07.502743 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:07.503667 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:07.503675 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:07.503694 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:07.504208 (+ 514us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:07.504284 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:07.504311 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:07.504421 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:07.504600 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 02:47:07.504899 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":129,"threads_started":1}
I20260811 02:47:07.506659 31326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56: No bootstrap required, opened a new log
I20260811 02:47:07.508965 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:47:07.511485 31326 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b920730b42439b8676583a45104e56" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39711 } }
I20260811 02:47:07.511729 31326 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:07.511790 31326 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0b920730b42439b8676583a45104e56, State: Initialized, Role: FOLLOWER
I20260811 02:47:07.512207 31326 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b920730b42439b8676583a45104e56" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39711 } }
I20260811 02:47:07.512324 31326 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:07.512408 31326 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:07.512542 31326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:07.513561 31326 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b920730b42439b8676583a45104e56" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39711 } }
I20260811 02:47:07.513904 31326 leader_election.cc:302] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0b920730b42439b8676583a45104e56; no voters:
I20260811 02:47:07.514269 31326 leader_election.cc:288] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:47:07.514447 31336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:07.514739 31336 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [term 1 LEADER]: Becoming Leader. State: Replica: d0b920730b42439b8676583a45104e56, State: Running, Role: LEADER
I20260811 02:47:07.515138 31336 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b920730b42439b8676583a45104e56" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39711 } }
I20260811 02:47:07.515718 31326 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:07.517163 31337 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0b920730b42439b8676583a45104e56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b920730b42439b8676583a45104e56" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39711 } } }
I20260811 02:47:07.517179 31341 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0b920730b42439b8676583a45104e56. Latest consensus state: current_term: 1 leader_uuid: "d0b920730b42439b8676583a45104e56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0b920730b42439b8676583a45104e56" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39711 } } }
I20260811 02:47:07.517320 31337 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:07.517321 31341 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56 [sys.catalog]: This master's current role is: LEADER
W20260811 02:47:07.520602 31358 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:47:07.520695 31358 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:47:07.520777 31352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:07.521729 31352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:07.527748 31352 catalog_manager.cc:1347] Generated new cluster ID: 9ee02a2460e14c97ada82fb7dc81b7fb
I20260811 02:47:07.527837 31352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:07.548602 31352 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:47:07.549639 31352 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:07.554494 31352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0b920730b42439b8676583a45104e56: Generated new TSK 0
I20260811 02:47:07.555356 31352 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 02:47:07.594138 31372 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 02:47:07.594319 31372 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 02:47:07.594913 31334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:07.576085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45976 (local address 127.21.236.190:39711)
0811 02:47:07.576227 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:07.576232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:07.576463 (+ 231us) server_negotiation.cc:407] Connection header received
0811 02:47:07.576661 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:07.576664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:07.576721 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:07.576835 (+ 114us) server_negotiation.cc:226] Negotiated authn=JWT
0811 02:47:07.577162 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:07.578002 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:07.578987 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:07.579387 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:07.579501 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:07.579922 (+ 421us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 02:47:07.594453 (+ 14531us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 02:47:07.594755 (+ 302us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45976: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 02:47:07.595000 31334 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45976: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 02:47:07.596247 31370 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:07.576018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:39711 (local address 127.0.0.1:45976)
0811 02:47:07.576346 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:07.576359 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:07.576505 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:07.576907 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:07.576910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:07.576927 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0811 02:47:07.577042 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:07.577047 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:07.578134 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:07.578137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:07.578861 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:07.578872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:07.579530 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:07.579533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:07.579608 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:07.579651 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 02:47:07.594716 (+ 15065us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 02:47:07.594945 (+ 229us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.236.190:39711: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":117,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.21.236.190:39711: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 02:47:07.597384 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 31223
2026-08-11T02:47:07Z chronyd exiting
I20260811 02:47:07.644696 22450 test_util.cc:175] -----------------------------------------------
I20260811 02:47:07.644794 22450 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786416395682572-22450-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (352 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 02:47:07.645987 22450 mini_oidc.cc:152] Starting JWKS server
I20260811 02:47:07.646888 22450 webserver.cc:460] Webserver started at https://127.0.0.1:40669/ using document root <none> and password file <none>
I20260811 02:47:07.647015 22450 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 02:47:07.647367 22450 webserver.cc:460] Webserver started at http://127.0.0.1:33535/ using document root <none> and password file <none>
2026-08-11T02:47:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:07Z Disabled control of system clock
I20260811 02:47:07.657559 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:46025
--webserver_interface=127.21.236.190
--webserver_port=0
--builtin_ntp_servers=127.21.236.148:33859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:46025
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40669/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/testchainca.cert with env {}
W20260811 02:47:07.777537 31379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:07.777837 31379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:07.777879 31379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:07.782078 31379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:07.782166 31379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:07.782181 31379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:07.782199 31379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:07.782215 31379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:47:07.782233 31379 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 02:47:07.782248 31379 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40669/jwks.json
I20260811 02:47:07.788089 31379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:46025
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:46025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40669/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:07.789178 31379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:07.790169 31379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:07.799350 31392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:07.799362 31391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:07.799373 31402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:07.799434 31379 server_base.cc:1034] running on GCE node
I20260811 02:47:07.800181 31379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:07.801306 31379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:07.802490 31379 hybrid_clock.cc:648] HybridClock initialized: now 1786416427802458 us; error 45 us; skew 500 ppm
I20260811 02:47:07.804548 31379 webserver.cc:460] Webserver started at http://127.21.236.190:45877/ using document root <none> and password file <none>
I20260811 02:47:07.805178 31379 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:07.805244 31379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:07.805469 31379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:07.807634 31379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "d0c69348d7414d468a61017f83be97b7"
format_stamp: "Formatted at 2026-08-11 02:47:07 on dist-test-slave-n326"
server_key: "70ae51987d3f3f6cc7c68c35e31daf8a"
server_key_iv: "9a1fd131eafde7f75b2e5cfe3cd6755f"
server_key_version: "encryptionkey@0"
I20260811 02:47:07.808212 31379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "d0c69348d7414d468a61017f83be97b7"
format_stamp: "Formatted at 2026-08-11 02:47:07 on dist-test-slave-n326"
server_key: "70ae51987d3f3f6cc7c68c35e31daf8a"
server_key_iv: "9a1fd131eafde7f75b2e5cfe3cd6755f"
server_key_version: "encryptionkey@0"
I20260811 02:47:07.812006 31379 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:47:07.814615 31415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:07.816058 31379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:47:07.816256 31379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "d0c69348d7414d468a61017f83be97b7"
format_stamp: "Formatted at 2026-08-11 02:47:07 on dist-test-slave-n326"
server_key: "70ae51987d3f3f6cc7c68c35e31daf8a"
server_key_iv: "9a1fd131eafde7f75b2e5cfe3cd6755f"
server_key_version: "encryptionkey@0"
I20260811 02:47:07.816393 31379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:07.836160 31379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:07.836920 31379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:07.837145 31379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:07.844678 31379 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:46025
I20260811 02:47:07.844694 31490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:46025 every 8 connection(s)
I20260811 02:47:07.845742 31379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:07.849283 31491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:07.854514 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 31379
I20260811 02:47:07.854682 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:47:07.854969 22450 external_mini_cluster.cc:1428] Setting key 5a847bb257151546edeca61fc93785a0
I20260811 02:47:07.855249 31491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7: Bootstrap starting.
I20260811 02:47:07.857769 31491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:07.858577 31491 log.cc:824] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:07.860579 31491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7: No bootstrap required, opened a new log
I20260811 02:47:07.864292 31491 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c69348d7414d468a61017f83be97b7" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46025 } }
I20260811 02:47:07.864548 31491 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:07.864629 31491 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0c69348d7414d468a61017f83be97b7, State: Initialized, Role: FOLLOWER
I20260811 02:47:07.864799 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:07.856518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54432 (local address 127.21.236.190:46025)
0811 02:47:07.857196 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:07.857210 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:07.857246 (+ 36us) server_negotiation.cc:407] Connection header received
0811 02:47:07.857868 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:07.857914 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:07.858473 (+ 559us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:07.858821 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:07.859306 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:07.860295 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:07.861293 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:07.861776 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:07.861965 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:07.862851 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:07.862856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:07.862870 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:07.863356 (+ 486us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:07.863418 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:07.863439 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:07.863546 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:07.863684 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 02:47:07.863946 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":151,"threads_started":1}
I20260811 02:47:07.865165 31491 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c69348d7414d468a61017f83be97b7" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46025 } }
I20260811 02:47:07.865339 31491 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:07.865432 31491 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:07.865535 31491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:07.866747 31491 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c69348d7414d468a61017f83be97b7" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46025 } }
I20260811 02:47:07.867158 31491 leader_election.cc:302] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0c69348d7414d468a61017f83be97b7; no voters:
I20260811 02:47:07.867552 31491 leader_election.cc:288] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:47:07.867625 31500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:07.867895 31500 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [term 1 LEADER]: Becoming Leader. State: Replica: d0c69348d7414d468a61017f83be97b7, State: Running, Role: LEADER
I20260811 02:47:07.868031 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:47:07.868222 31500 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c69348d7414d468a61017f83be97b7" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46025 } }
I20260811 02:47:07.868899 31491 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:07.869813 31507 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0c69348d7414d468a61017f83be97b7. Latest consensus state: current_term: 1 leader_uuid: "d0c69348d7414d468a61017f83be97b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c69348d7414d468a61017f83be97b7" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46025 } } }
I20260811 02:47:07.869952 31507 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:07.870620 31503 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0c69348d7414d468a61017f83be97b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c69348d7414d468a61017f83be97b7" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46025 } } }
I20260811 02:47:07.871301 31521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:07.871369 31503 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:07.874025 31521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:07.880057 31521 catalog_manager.cc:1347] Generated new cluster ID: e47dabaf04974a58a491902c7a6f7727
I20260811 02:47:07.880144 31521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:07.909744 31521 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:47:07.910816 31521 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:07.919263 31521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0c69348d7414d468a61017f83be97b7: Generated new TSK 0
I20260811 02:47:07.920017 31521 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 02:47:07.947546 31545 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 02:47:07.947646 31545 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 02:47:07.948303 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:07.931954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54434 (local address 127.21.236.190:46025)
0811 02:47:07.932115 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:07.932120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:07.932363 (+ 243us) server_negotiation.cc:407] Connection header received
0811 02:47:07.932571 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:07.932574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:07.932624 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:07.932717 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0811 02:47:07.932998 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:07.933553 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:07.934871 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:07.935184 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:07.935268 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:07.935701 (+ 433us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 02:47:07.947973 (+ 12272us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 02:47:07.948193 (+ 220us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54434: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 02:47:07.948405 31495 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54434: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 02:47:07.948616 31543 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:07.931904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:46025 (local address 127.0.0.1:54434)
0811 02:47:07.932261 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:07.932276 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:07.932431 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:07.932750 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:07.932753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:07.932761 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 02:47:07.932881 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:07.932888 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:07.933672 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:07.933676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:07.934742 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:07.934753 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:07.935301 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:07.935305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:07.935465 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:07.935496 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 02:47:07.948155 (+ 12659us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 02:47:07.948428 (+ 273us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.236.190:46025: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40669/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":156,"threads_started":1}
I20260811 02:47:07.949213 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 31379
2026-08-11T02:47:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (353 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 02:47:07.999737 22450 mini_oidc.cc:152] Starting JWKS server
I20260811 02:47:08.000659 22450 webserver.cc:460] Webserver started at https://127.0.0.1:32981/ using document root <none> and password file <none>
I20260811 02:47:08.000797 22450 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 02:47:08.001075 22450 webserver.cc:460] Webserver started at http://127.0.0.1:36209/ using document root <none> and password file <none>
2026-08-11T02:47:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:08Z Disabled control of system clock
I20260811 02:47:08.011567 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:41083
--webserver_interface=127.21.236.190
--webserver_port=0
--builtin_ntp_servers=127.21.236.148:39343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:41083
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32981/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 02:47:08.125535 31553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:08.125840 31553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:08.125882 31553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:08.129880 31553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:08.129952 31553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:08.129968 31553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:08.129985 31553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:08.130003 31553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:47:08.130019 31553 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 02:47:08.130033 31553 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32981/jwks.json
I20260811 02:47:08.134884 31553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:41083
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:41083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32981/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:08.136118 31553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:08.137617 31553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:08.146849 31559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:08.146981 31553 server_base.cc:1034] running on GCE node
W20260811 02:47:08.146857 31562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:08.146872 31565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:08.147866 31553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:08.148996 31553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:08.150173 31553 hybrid_clock.cc:648] HybridClock initialized: now 1786416428150157 us; error 31 us; skew 500 ppm
I20260811 02:47:08.156307 31553 webserver.cc:460] Webserver started at http://127.21.236.190:39079/ using document root <none> and password file <none>
I20260811 02:47:08.157009 31553 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:08.157114 31553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:08.157351 31553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:08.159595 31553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "19b0da703d18467a9128eef1c8df9c06"
format_stamp: "Formatted at 2026-08-11 02:47:08 on dist-test-slave-n326"
server_key: "f304f8e4c9b20afdd76d531ba132921f"
server_key_iv: "3ead63e423ef0d80b27d3ea6c29c37f4"
server_key_version: "encryptionkey@0"
I20260811 02:47:08.160132 31553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "19b0da703d18467a9128eef1c8df9c06"
format_stamp: "Formatted at 2026-08-11 02:47:08 on dist-test-slave-n326"
server_key: "f304f8e4c9b20afdd76d531ba132921f"
server_key_iv: "3ead63e423ef0d80b27d3ea6c29c37f4"
server_key_version: "encryptionkey@0"
I20260811 02:47:08.164091 31553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:47:08.166707 31574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:08.168236 31553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:47:08.168399 31553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "19b0da703d18467a9128eef1c8df9c06"
format_stamp: "Formatted at 2026-08-11 02:47:08 on dist-test-slave-n326"
server_key: "f304f8e4c9b20afdd76d531ba132921f"
server_key_iv: "3ead63e423ef0d80b27d3ea6c29c37f4"
server_key_version: "encryptionkey@0"
I20260811 02:47:08.168560 31553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:08.198892 31553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:08.199827 31553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:08.200119 31553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:08.209761 31553 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:41083
I20260811 02:47:08.209796 31669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:41083 every 8 connection(s)
I20260811 02:47:08.210788 31553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:08.214190 31670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:08.218307 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 31553
I20260811 02:47:08.218469 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:47:08.218791 22450 external_mini_cluster.cc:1428] Setting key d92ed2cee39820d7fd4779318b18b835
I20260811 02:47:08.220770 31670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06: Bootstrap starting.
I20260811 02:47:08.223277 31670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:08.224058 31670 log.cc:824] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:08.225998 31670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06: No bootstrap required, opened a new log
I20260811 02:47:08.229422 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:08.220625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39568 (local address 127.21.236.190:41083)
0811 02:47:08.221464 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:08.221483 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:08.221523 (+ 40us) server_negotiation.cc:407] Connection header received
0811 02:47:08.222263 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:08.222282 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:08.222739 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:08.223161 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:08.223727 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:08.224867 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:08.225864 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:08.226442 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:08.226643 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:08.227613 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:08.227621 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:08.227641 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:08.228024 (+ 383us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:08.228081 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:08.228110 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:08.228225 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:08.228377 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 02:47:08.228673 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":180,"threads_started":1}
I20260811 02:47:08.229849 31670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b0da703d18467a9128eef1c8df9c06" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41083 } }
I20260811 02:47:08.230098 31670 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:08.230187 31670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b0da703d18467a9128eef1c8df9c06, State: Initialized, Role: FOLLOWER
I20260811 02:47:08.230681 31670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b0da703d18467a9128eef1c8df9c06" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41083 } }
I20260811 02:47:08.230839 31670 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:08.230901 31670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:08.230995 31670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:08.232210 31670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b0da703d18467a9128eef1c8df9c06" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41083 } }
I20260811 02:47:08.232550 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:47:08.232585 31670 leader_election.cc:302] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19b0da703d18467a9128eef1c8df9c06; no voters:
I20260811 02:47:08.232933 31670 leader_election.cc:288] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:47:08.233013 31677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:08.233193 31677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [term 1 LEADER]: Becoming Leader. State: Replica: 19b0da703d18467a9128eef1c8df9c06, State: Running, Role: LEADER
I20260811 02:47:08.233496 31677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b0da703d18467a9128eef1c8df9c06" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41083 } }
I20260811 02:47:08.234215 31670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:08.235124 31680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19b0da703d18467a9128eef1c8df9c06. Latest consensus state: current_term: 1 leader_uuid: "19b0da703d18467a9128eef1c8df9c06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b0da703d18467a9128eef1c8df9c06" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41083 } } }
I20260811 02:47:08.235091 31678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19b0da703d18467a9128eef1c8df9c06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19b0da703d18467a9128eef1c8df9c06" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41083 } } }
I20260811 02:47:08.235260 31680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:08.235261 31678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:08.235674 31689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:08.239265 31689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:08.244768 31689 catalog_manager.cc:1347] Generated new cluster ID: 8d0e220d7982456b91d0c68cb4396998
I20260811 02:47:08.244850 31689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:08.277638 31689 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:47:08.278635 31689 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:08.283438 31689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19b0da703d18467a9128eef1c8df9c06: Generated new TSK 0
I20260811 02:47:08.284375 31689 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:47:08.303349 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:08.297805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39584 (local address 127.21.236.190:41083)
0811 02:47:08.297964 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:08.297969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:08.297984 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:47:08.298068 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:08.298071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:08.298122 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:08.298225 (+ 103us) server_negotiation.cc:226] Negotiated authn=JWT
0811 02:47:08.298534 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:08.299207 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:08.300506 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:08.300753 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:08.300836 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:08.301217 (+ 381us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 02:47:08.302839 (+ 1622us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32981/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 02:47:08.303182 (+ 343us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39584: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32981/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 02:47:08.303435 31673 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39584: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32981/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 02:47:08.305374 31714 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:08.297414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41083 (local address 127.0.0.1:39584)
0811 02:47:08.297783 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:08.297799 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:08.297909 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:08.298242 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:08.298245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:08.298253 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 02:47:08.298387 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:08.298394 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:08.299339 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:08.299342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:08.300357 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:08.300378 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:08.300876 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:08.300880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:08.300971 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:08.301001 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 02:47:08.303060 (+ 2059us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32981/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 02:47:08.305172 (+ 2112us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.236.190:41083: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32981/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":1870,"thread_start_us":100,"threads_started":1}
I20260811 02:47:08.306056 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 31553
E20260811 02:47:08.329427 31716 webserver.cc:533] Webserver: sslize failed:
E20260811 02:47:08.329520 31716 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T02:47:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (359 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T02:47:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:08Z Disabled control of system clock
I20260811 02:47:08.368691 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:41427
--webserver_interface=127.21.236.190
--webserver_port=0
--builtin_ntp_servers=127.21.236.148:46551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:41427
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/testchainca.cert with env {}
W20260811 02:47:08.482264 31725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:08.482543 31725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:08.482591 31725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:08.486645 31725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:08.486706 31725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:08.486722 31725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:08.486739 31725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:08.486755 31725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 02:47:08.486812 31725 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 02:47:08.486840 31725 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 02:47:08.492347 31725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:46551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:41427
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:41427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:08.493599 31725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:08.494702 31725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:08.503473 31736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:08.503492 31738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:08.503473 31732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:08.503901 31725 server_base.cc:1034] running on GCE node
I20260811 02:47:08.504539 31725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:08.505717 31725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:08.506912 31725 hybrid_clock.cc:648] HybridClock initialized: now 1786416428506894 us; error 39 us; skew 500 ppm
I20260811 02:47:08.508962 31725 webserver.cc:460] Webserver started at http://127.21.236.190:46491/ using document root <none> and password file <none>
I20260811 02:47:08.509593 31725 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:08.509694 31725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:08.509932 31725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:08.512145 31725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "5cc76fd320064266a367a99c375379e6"
format_stamp: "Formatted at 2026-08-11 02:47:08 on dist-test-slave-n326"
server_key: "4600b8c12a79e020562959415d679180"
server_key_iv: "1f5372633c3eed61dcf1d034ef994a71"
server_key_version: "encryptionkey@0"
I20260811 02:47:08.512754 31725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "5cc76fd320064266a367a99c375379e6"
format_stamp: "Formatted at 2026-08-11 02:47:08 on dist-test-slave-n326"
server_key: "4600b8c12a79e020562959415d679180"
server_key_iv: "1f5372633c3eed61dcf1d034ef994a71"
server_key_version: "encryptionkey@0"
I20260811 02:47:08.516831 31725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:47:08.519701 31749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:08.521042 31725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:47:08.521200 31725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "5cc76fd320064266a367a99c375379e6"
format_stamp: "Formatted at 2026-08-11 02:47:08 on dist-test-slave-n326"
server_key: "4600b8c12a79e020562959415d679180"
server_key_iv: "1f5372633c3eed61dcf1d034ef994a71"
server_key_version: "encryptionkey@0"
I20260811 02:47:08.521325 31725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:08.533660 31725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:08.534386 31725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:08.534607 31725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:08.541836 31725 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:41427
I20260811 02:47:08.541818 31842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:41427 every 8 connection(s)
I20260811 02:47:08.542865 31725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:08.544931 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 31725
I20260811 02:47:08.545074 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:47:08.545372 22450 external_mini_cluster.cc:1428] Setting key 6c2a92eb0053ca0a7c03736b774dbbaa
I20260811 02:47:08.546247 31846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:08.552248 31846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6: Bootstrap starting.
I20260811 02:47:08.554580 31846 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:08.554790 31851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:08.546880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56910 (local address 127.21.236.190:41427)
0811 02:47:08.547568 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:08.547583 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:08.547619 (+ 36us) server_negotiation.cc:407] Connection header received
0811 02:47:08.548279 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:08.548295 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:08.548703 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:08.549070 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:08.549486 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:08.550461 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:08.551455 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:08.551931 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:08.552131 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:08.553051 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:08.553056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:08.553067 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:08.553450 (+ 383us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:08.553509 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:08.553530 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:08.553640 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:08.553761 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 02:47:08.554038 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":171,"threads_started":1}
I20260811 02:47:08.555534 31846 log.cc:824] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:08.557624 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:47:08.557706 31846 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6: No bootstrap required, opened a new log
I20260811 02:47:08.561408 31846 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc76fd320064266a367a99c375379e6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41427 } }
I20260811 02:47:08.561625 31846 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:08.561708 31846 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cc76fd320064266a367a99c375379e6, State: Initialized, Role: FOLLOWER
I20260811 02:47:08.562155 31846 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc76fd320064266a367a99c375379e6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41427 } }
I20260811 02:47:08.562296 31846 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:08.562366 31846 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:08.562482 31846 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:08.563584 31846 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc76fd320064266a367a99c375379e6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41427 } }
I20260811 02:47:08.563925 31846 leader_election.cc:302] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cc76fd320064266a367a99c375379e6; no voters:
I20260811 02:47:08.564227 31846 leader_election.cc:288] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:47:08.564369 31855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:08.564626 31855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [term 1 LEADER]: Becoming Leader. State: Replica: 5cc76fd320064266a367a99c375379e6, State: Running, Role: LEADER
I20260811 02:47:08.564914 31855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc76fd320064266a367a99c375379e6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41427 } }
I20260811 02:47:08.565418 31846 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:08.566640 31859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cc76fd320064266a367a99c375379e6. Latest consensus state: current_term: 1 leader_uuid: "5cc76fd320064266a367a99c375379e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc76fd320064266a367a99c375379e6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41427 } } }
I20260811 02:47:08.566792 31859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:08.567203 31857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cc76fd320064266a367a99c375379e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cc76fd320064266a367a99c375379e6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41427 } } }
I20260811 02:47:08.567373 31857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:08.567904 31870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:08.570477 31870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:08.575774 31870 catalog_manager.cc:1347] Generated new cluster ID: 844be0613949496db293789e64a8d3b3
I20260811 02:47:08.575845 31870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:08.581795 31870 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:47:08.582644 31870 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:08.589804 31870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cc76fd320064266a367a99c375379e6: Generated new TSK 0
I20260811 02:47:08.590490 31870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:47:08.699054 31851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:08.597540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56924 (local address 127.21.236.190:41427)
0811 02:47:08.597684 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:08.597689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:08.597884 (+ 195us) server_negotiation.cc:407] Connection header received
0811 02:47:08.598055 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:08.598057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:08.598103 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:08.598194 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0811 02:47:08.598468 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:08.598977 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:08.600296 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:08.600510 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:08.600590 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:08.600942 (+ 352us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 02:47:08.698642 (+ 97700us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 02:47:08.698915 (+ 273us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56924: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 02:47:08.699239 31851 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56924: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 02:47:08.699580 31899 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:08.597459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41427 (local address 127.0.0.1:56924)
0811 02:47:08.597785 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:08.597801 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:08.597930 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:08.598222 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:08.598225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:08.598233 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 02:47:08.598361 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:08.598368 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:08.599152 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:08.599156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:08.600177 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:08.600189 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:08.600635 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:08.600639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:08.600735 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:08.600768 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 02:47:08.698910 (+ 98142us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 02:47:08.699197 (+ 287us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.236.190:41427: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":121,"threads_started":1}
I20260811 02:47:08.700350 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 31725
2026-08-11T02:47:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (372 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (130 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (128 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:47:09 dist-test-slave-n326 krb5kdc[31918](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:47:09 dist-test-slave-n326 krb5kdc[31918](info): set up 2 sockets
Aug 11 02:47:09 dist-test-slave-n326 krb5kdc[31918](info): commencing operation
krb5kdc: starting...
W20260811 02:47:10.273708 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:47:10 dist-test-slave-n326 krb5kdc[31918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:47:10.414726 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:36857
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:36857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:10.530615 31946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:10.530918 31946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:10.530961 31946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:10.535182 31946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:10.535259 31946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:10.535280 31946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:10.535297 31946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:10.535313 31946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:47:10.540838 31946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:36857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:36857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:10.542142 31946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:10.543262 31946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:10.552251 31956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:10.552284 31967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:10.552342 31946 server_base.cc:1034] running on GCE node
W20260811 02:47:10.552275 31963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:10.553052 31946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:10.554162 31946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:10.555347 31946 hybrid_clock.cc:648] HybridClock initialized: now 1786416430555313 us; error 46 us; skew 500 ppm
Aug 11 02:47:10 dist-test-slave-n326 krb5kdc[31918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416430, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:10.558167 31946 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:47:10.559154 31946 webserver.cc:460] Webserver started at http://127.21.236.190:35703/ using document root <none> and password file <none>
I20260811 02:47:10.559751 31946 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:10.559844 31946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:10.560063 31946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:10.562171 31946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "5d42e846a67842dcb8cb87287d99f5bb"
format_stamp: "Formatted at 2026-08-11 02:47:10 on dist-test-slave-n326"
server_key: "655df4f670617ed1a732d158632038cc"
server_key_iv: "9d787e9b37083c5f7d67e46a8e32ed94"
server_key_version: "encryptionkey@0"
I20260811 02:47:10.562781 31946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "5d42e846a67842dcb8cb87287d99f5bb"
format_stamp: "Formatted at 2026-08-11 02:47:10 on dist-test-slave-n326"
server_key: "655df4f670617ed1a732d158632038cc"
server_key_iv: "9d787e9b37083c5f7d67e46a8e32ed94"
server_key_version: "encryptionkey@0"
I20260811 02:47:10.566787 31946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:47:10.569483 31976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:10.570746 31946 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:47:10.570909 31946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "5d42e846a67842dcb8cb87287d99f5bb"
format_stamp: "Formatted at 2026-08-11 02:47:10 on dist-test-slave-n326"
server_key: "655df4f670617ed1a732d158632038cc"
server_key_iv: "9d787e9b37083c5f7d67e46a8e32ed94"
server_key_version: "encryptionkey@0"
I20260811 02:47:10.571033 31946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:10.590840 31946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:10.594084 31946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:10.594298 31946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:10.602238 31946 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:36857
I20260811 02:47:10.602263 32061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:36857 every 8 connection(s)
I20260811 02:47:10.603389 31946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:10.606279 32062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:10.611950 32062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb: Bootstrap starting.
I20260811 02:47:10.613220 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 31946
I20260811 02:47:10.613369 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:47:10.613713 22450 external_mini_cluster.cc:1428] Setting key 4f77dedc5a4b54fb8d18fb72490a12e6
I20260811 02:47:10.614655 32062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:10.615609 32062 log.cc:824] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:10.617576 32062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb: No bootstrap required, opened a new log
I20260811 02:47:10.621340 32062 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d42e846a67842dcb8cb87287d99f5bb" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36857 } }
I20260811 02:47:10.621637 32062 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:10.621727 32062 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d42e846a67842dcb8cb87287d99f5bb, State: Initialized, Role: FOLLOWER
I20260811 02:47:10.622205 32062 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d42e846a67842dcb8cb87287d99f5bb" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36857 } }
I20260811 02:47:10.622394 32062 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:10.622465 32062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:10.622577 32062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [term 0 FOLLOWER]: Advancing to term 1
Aug 11 02:47:10 dist-test-slave-n326 krb5kdc[31918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:10.623675 32062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d42e846a67842dcb8cb87287d99f5bb" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36857 } }
I20260811 02:47:10.624061 32062 leader_election.cc:302] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d42e846a67842dcb8cb87287d99f5bb; no voters:
I20260811 02:47:10.624504 32062 leader_election.cc:288] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:47:10.624590 32070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:10.624792 32070 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [term 1 LEADER]: Becoming Leader. State: Replica: 5d42e846a67842dcb8cb87287d99f5bb, State: Running, Role: LEADER
I20260811 02:47:10.625164 32070 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d42e846a67842dcb8cb87287d99f5bb" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36857 } }
I20260811 02:47:10.625926 32062 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:10.627378 32072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d42e846a67842dcb8cb87287d99f5bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d42e846a67842dcb8cb87287d99f5bb" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36857 } } }
I20260811 02:47:10.627516 32072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:10.627763 32075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d42e846a67842dcb8cb87287d99f5bb. Latest consensus state: current_term: 1 leader_uuid: "5d42e846a67842dcb8cb87287d99f5bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d42e846a67842dcb8cb87287d99f5bb" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36857 } } }
I20260811 02:47:10.628108 32075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:10.629966 32067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:10.614913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56562 (local address 127.21.236.190:36857)
0811 02:47:10.615399 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:10.615415 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:10.615453 (+ 38us) server_negotiation.cc:407] Connection header received
0811 02:47:10.616183 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:10.616203 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:10.616542 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:10.616887 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:10.617732 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:10.618695 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:10.619658 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:10.620142 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:10.620320 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:10.623521 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:10.623527 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:10.623540 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:10.623571 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:10.625701 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:10.626309 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:10.626318 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:10.626326 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:10.626491 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:10.626854 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:10.626858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:10.626860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:10.627461 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:10.627676 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:10.628659 (+ 983us) server_negotiation.cc:299] Negotiation successful
0811 02:47:10.628985 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":329,"thread_start_us":159,"threads_started":1}
W20260811 02:47:10.631742 32104 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:47:10.631819 32104 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:47:10.631897 32098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:10.632651 32098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:10.638808 32098 catalog_manager.cc:1347] Generated new cluster ID: 1900db25608c45f9aff51ab9b5c9f43d
I20260811 02:47:10.638902 32098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:10.658087 32098 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:47:10.659158 32098 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:10.681317 32098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d42e846a67842dcb8cb87287d99f5bb: Generated new TSK 0
I20260811 02:47:10.682166 32098 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:47:10.727785 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:36857
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:10.852149 32115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:10.852469 32115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:10.852514 32115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:10.856696 32115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:10.856762 32115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:10.856822 32115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:47:10.862151 32115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:36857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:10.863291 32115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:10.864303 32115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:10.874454 32129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:10.874536 32126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:10.874454 32122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:10.874711 32115 server_base.cc:1034] running on GCE node
I20260811 02:47:10.875309 32115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:10.876060 32115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:10.877228 32115 hybrid_clock.cc:648] HybridClock initialized: now 1786416430877215 us; error 24 us; skew 500 ppm
Aug 11 02:47:10 dist-test-slave-n326 krb5kdc[31918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416430, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:10.880038 32115 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:47:10.880931 32115 webserver.cc:460] Webserver started at http://127.21.236.129:38375/ using document root <none> and password file <none>
I20260811 02:47:10.881515 32115 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:10.881635 32115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:10.881876 32115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:10.883975 32115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "c04526c918bf45f1a4729a487b6cda0e"
format_stamp: "Formatted at 2026-08-11 02:47:10 on dist-test-slave-n326"
server_key: "c923b0c78e1694dfb6ef686ef3207112"
server_key_iv: "fc690a2d40d4721cf2ab70eca7cb77ed"
server_key_version: "encryptionkey@0"
I20260811 02:47:10.884538 32115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "c04526c918bf45f1a4729a487b6cda0e"
format_stamp: "Formatted at 2026-08-11 02:47:10 on dist-test-slave-n326"
server_key: "c923b0c78e1694dfb6ef686ef3207112"
server_key_iv: "fc690a2d40d4721cf2ab70eca7cb77ed"
server_key_version: "encryptionkey@0"
I20260811 02:47:10.888417 32115 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:47:10.890894 32143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:10.892192 32115 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:47:10.892339 32115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "c04526c918bf45f1a4729a487b6cda0e"
format_stamp: "Formatted at 2026-08-11 02:47:10 on dist-test-slave-n326"
server_key: "c923b0c78e1694dfb6ef686ef3207112"
server_key_iv: "fc690a2d40d4721cf2ab70eca7cb77ed"
server_key_version: "encryptionkey@0"
I20260811 02:47:10.892454 32115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:10.904904 32115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:10.907946 32115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:10.908156 32115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:10.908771 32115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:10.909621 32115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:10.909693 32115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:10.909762 32115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:10.909802 32115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:10.919428 32115 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:37257
I20260811 02:47:10.919510 32331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:37257 every 8 connection(s)
I20260811 02:47:10.920392 32115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:47:10.926075 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 32115
I20260811 02:47:10.926218 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:47:10.926512 22450 external_mini_cluster.cc:1428] Setting key e3099aeda43cbef59cc54244d90a5b38
Aug 11 02:47:10 dist-test-slave-n326 krb5kdc[31918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416430, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:10.936167 32067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:10.922340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:39857 (local address 127.21.236.190:36857)
0811 02:47:10.922541 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:10.922546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:10.923370 (+ 824us) server_negotiation.cc:407] Connection header received
0811 02:47:10.924763 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:10.924766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:10.924827 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:10.924936 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:10.926190 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:10.926953 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:10.928622 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:10.928852 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:10.928937 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:10.932140 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:10.932144 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:10.932145 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:10.932171 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:10.933913 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:10.934467 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:10.934470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:10.934472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:10.934565 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:10.934925 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:10.934929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:10.934930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:10.935200 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:10.935310 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:10.935836 (+ 526us) server_negotiation.cc:299] Negotiation successful
0811 02:47:10.935983 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 02:47:10.937003 32335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:10.922690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:36857 (local address 127.21.236.129:39857)
0811 02:47:10.923208 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:10.923252 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:10.924554 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:10.925116 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:10.925122 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:10.925554 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:10.926038 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:10.926052 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:10.927180 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:10.927184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:10.928482 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:10.928497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:10.929011 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:10.929018 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:10.929267 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:10.929925 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:10.929952 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:10.931958 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:10.934057 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:10.934063 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:10.934076 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:10.934336 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:10.934662 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:10.934666 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:10.934668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:10.934813 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:10.935311 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:10.935320 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:10.935581 (+ 261us) client_negotiation.cc:769] Sending connection context
0811 02:47:10.935775 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 02:47:10.936124 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
I20260811 02:47:10.938423 32332 heartbeater.cc:347] Connected to a master server at 127.21.236.190:36857
I20260811 02:47:10.938715 32332 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:10.939262 32332 heartbeater.cc:511] Master 127.21.236.190:36857 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
I20260811 02:47:10.941032 32002 ts_manager.cc:194] Registered new tserver with Master: c04526c918bf45f1a4729a487b6cda0e (127.21.236.129:37257)
I20260811 02:47:10.942147 32002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:39857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:47:10.958873 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:36857
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:11.076853 32338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:11.077104 32338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:11.077144 32338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:11.081143 32338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:11.081197 32338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:11.081297 32338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:47:11.086678 32338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:36857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:11.087988 32338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:11.089123 32338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:11.099175 32349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:11.099187 32347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:11.099273 32346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:11.099519 32338 server_base.cc:1034] running on GCE node
I20260811 02:47:11.099956 32338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:11.100683 32338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:11.101903 32338 hybrid_clock.cc:648] HybridClock initialized: now 1786416431101855 us; error 64 us; skew 500 ppm
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[31918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416431, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:11.104898 32338 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:47:11.105801 32338 webserver.cc:460] Webserver started at http://127.21.236.130:33309/ using document root <none> and password file <none>
I20260811 02:47:11.106436 32338 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:11.106536 32338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:11.106775 32338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:11.108888 32338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "d79b806ccb3f4242ac73bcea89002b80"
format_stamp: "Formatted at 2026-08-11 02:47:11 on dist-test-slave-n326"
server_key: "700c51a3da8024a03fe04816382e4efe"
server_key_iv: "3def7a04c82d3f1af5c5b14d34c100bc"
server_key_version: "encryptionkey@0"
I20260811 02:47:11.109472 32338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "d79b806ccb3f4242ac73bcea89002b80"
format_stamp: "Formatted at 2026-08-11 02:47:11 on dist-test-slave-n326"
server_key: "700c51a3da8024a03fe04816382e4efe"
server_key_iv: "3def7a04c82d3f1af5c5b14d34c100bc"
server_key_version: "encryptionkey@0"
I20260811 02:47:11.113253 32338 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 02:47:11.115747 32370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:11.117012 32338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:47:11.117153 32338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "d79b806ccb3f4242ac73bcea89002b80"
format_stamp: "Formatted at 2026-08-11 02:47:11 on dist-test-slave-n326"
server_key: "700c51a3da8024a03fe04816382e4efe"
server_key_iv: "3def7a04c82d3f1af5c5b14d34c100bc"
server_key_version: "encryptionkey@0"
I20260811 02:47:11.117267 32338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:11.147460 32338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:11.151047 32338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:11.151295 32338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:11.151955 32338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:11.152858 32338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:11.152932 32338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:11.153000 32338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:11.153048 32338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:11.162377 32338 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:38553
I20260811 02:47:11.162432 32564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:38553 every 8 connection(s)
I20260811 02:47:11.163386 32338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:47:11.167306 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 32338
I20260811 02:47:11.167424 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:47:11.167711 22450 external_mini_cluster.cc:1428] Setting key 5a267b89f0aa0e8a15ca623c120464d4
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[31918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416431, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:11.180012 32067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:11.165321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:35135 (local address 127.21.236.190:36857)
0811 02:47:11.165468 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:11.165472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:11.166352 (+ 880us) server_negotiation.cc:407] Connection header received
0811 02:47:11.168138 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:11.168142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:11.168206 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:11.168311 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:11.169813 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:11.170411 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:11.172001 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:11.172424 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:11.172565 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:11.175607 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:11.175615 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:11.175622 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:11.175664 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:11.177594 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:11.178162 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:11.178166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:11.178169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:11.178281 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:11.178653 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:11.178661 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:11.178664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:11.178945 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:11.179083 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:11.179669 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 02:47:11.179822 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
I20260811 02:47:11.180747 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:11.165750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:36857 (local address 127.21.236.130:35135)
0811 02:47:11.166200 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:11.166242 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:11.167847 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:11.168515 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:11.168560 (+ 45us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:11.168949 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:11.169632 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:11.169653 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:11.170565 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:11.170572 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:11.171869 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:11.171880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:11.172612 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:11.172615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:11.172752 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:11.173387 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:11.173404 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:11.175424 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:11.177727 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:11.177732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:11.177744 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:11.178025 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:11.178392 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:11.178394 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:11.178396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:11.178530 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:11.179113 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:11.179119 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:11.179375 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 02:47:11.179536 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 02:47:11.179803 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":122,"threads_started":1}
I20260811 02:47:11.182106 32567 heartbeater.cc:347] Connected to a master server at 127.21.236.190:36857
I20260811 02:47:11.182447 32567 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:11.183048 32567 heartbeater.cc:511] Master 127.21.236.190:36857 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
I20260811 02:47:11.184309 32002 ts_manager.cc:194] Registered new tserver with Master: d79b806ccb3f4242ac73bcea89002b80 (127.21.236.130:38553)
I20260811 02:47:11.185189 32002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:35135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:47:11.204260 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:36857
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:11.319823 32575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:11.320210 32575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:11.320298 32575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:11.324602 32575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:11.324667 32575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:11.324757 32575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:47:11.330420 32575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:36857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:11.331647 32575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:11.332711 32575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:11.342543 32593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:11.342571 32590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:11.342566 32588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:11.342907 32575 server_base.cc:1034] running on GCE node
I20260811 02:47:11.343433 32575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:11.344157 32575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:11.345352 32575 hybrid_clock.cc:648] HybridClock initialized: now 1786416431345312 us; error 50 us; skew 500 ppm
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[31918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416431, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:11.348312 32575 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:47:11.349255 32575 webserver.cc:460] Webserver started at http://127.21.236.131:39051/ using document root <none> and password file <none>
I20260811 02:47:11.349843 32575 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:11.349939 32575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:11.350183 32575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:11.352298 32575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "0354ca9e773e4dffbe6b41d9d9b46124"
format_stamp: "Formatted at 2026-08-11 02:47:11 on dist-test-slave-n326"
server_key: "9972312cbbbb4b49ba563f292dd7e6b0"
server_key_iv: "740bcb263ffef6e024534243d8690c5e"
server_key_version: "encryptionkey@0"
I20260811 02:47:11.352865 32575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "0354ca9e773e4dffbe6b41d9d9b46124"
format_stamp: "Formatted at 2026-08-11 02:47:11 on dist-test-slave-n326"
server_key: "9972312cbbbb4b49ba563f292dd7e6b0"
server_key_iv: "740bcb263ffef6e024534243d8690c5e"
server_key_version: "encryptionkey@0"
I20260811 02:47:11.356652 32575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:47:11.359256 32601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:11.360500 32575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:47:11.360638 32575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "0354ca9e773e4dffbe6b41d9d9b46124"
format_stamp: "Formatted at 2026-08-11 02:47:11 on dist-test-slave-n326"
server_key: "9972312cbbbb4b49ba563f292dd7e6b0"
server_key_iv: "740bcb263ffef6e024534243d8690c5e"
server_key_version: "encryptionkey@0"
I20260811 02:47:11.360756 32575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:11.389909 32575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:11.393376 32575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:11.393607 32575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:11.394229 32575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:11.395166 32575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:11.395234 32575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:11.395309 32575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:11.395351 32575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:11.404906 32575 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:44479
I20260811 02:47:11.404939 313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:44479 every 8 connection(s)
I20260811 02:47:11.405864 32575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:47:11.412834 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 32575
I20260811 02:47:11.413028 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:47:11.413347 22450 external_mini_cluster.cc:1428] Setting key b3581b0691916163907c150307fdcc9a
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[31918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416431, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:11.421228 32067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:11.407929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:55651 (local address 127.21.236.190:36857)
0811 02:47:11.408092 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:11.408096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:11.409043 (+ 947us) server_negotiation.cc:407] Connection header received
0811 02:47:11.410670 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:11.410673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:11.410739 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:11.410861 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:11.412211 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:11.412813 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:11.414248 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:11.414568 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:11.414695 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:11.417439 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:11.417442 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:11.417444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:11.417470 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:11.418917 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:11.419546 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:11.419549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:11.419550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:11.419638 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:11.419965 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:11.419968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:11.419969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:11.420163 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:11.420239 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:11.420976 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 02:47:11.421103 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 02:47:11.422214 317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:11.408323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:36857 (local address 127.21.236.131:55651)
0811 02:47:11.408859 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:11.408916 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:11.410436 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:11.411053 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:11.411108 (+ 55us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:11.411531 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:11.412047 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:11.412062 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:11.412968 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:11.412977 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:11.414105 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:11.414116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:11.414759 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:11.414765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:11.414956 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:11.415693 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:11.415717 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:11.417271 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:11.419057 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:11.419115 (+ 58us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:11.419135 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:11.419439 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:11.419729 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:11.419732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:11.419733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:11.419874 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:11.420272 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:11.420277 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:11.420709 (+ 432us) client_negotiation.cc:769] Sending connection context
0811 02:47:11.420875 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 02:47:11.421160 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":136,"threads_started":1}
I20260811 02:47:11.423496 314 heartbeater.cc:347] Connected to a master server at 127.21.236.190:36857
I20260811 02:47:11.423781 314 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:11.424330 314 heartbeater.cc:511] Master 127.21.236.190:36857 requested a full tablet report, sending...
I20260811 02:47:11.425536 32002 ts_manager.cc:194] Registered new tserver with Master: 0354ca9e773e4dffbe6b41d9d9b46124 (127.21.236.131:44479)
I20260811 02:47:11.426350 32002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:55651
I20260811 02:47:11.428072 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/test-admin.keytab.
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[31918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:11.445506 22450 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 02:47:11.448698 324 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:11.447150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59984 (local address 127.21.236.129:37257)
0811 02:47:11.447525 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:11.447531 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:11.447557 (+ 26us) server_negotiation.cc:407] Connection header received
0811 02:47:11.447610 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:11.447616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:11.447817 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:11.447960 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:11.448533 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59984: BlockingRecv error: recv got EOF from 127.0.0.1:59984 (error 108)
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":121,"threads_started":1}
W20260811 02:47:11.449157 22450 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/krb5cc)
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[31918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:11.450265 22450 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[31918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:11.451400 22450 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[31918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.129@KRBTEST.COM
I20260811 02:47:11.461908 324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:11.452332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59994 (local address 127.21.236.129:37257)
0811 02:47:11.452485 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:11.452488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:11.452593 (+ 105us) server_negotiation.cc:407] Connection header received
0811 02:47:11.452696 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:11.452698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:11.452742 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:11.452809 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:11.453342 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:11.454102 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:11.454996 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:11.455446 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:11.455530 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:11.458012 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:11.458024 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:11.458038 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:11.458063 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:11.459899 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:11.460414 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:11.460420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:11.460421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:11.460507 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:11.460823 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:11.460828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:11.460830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:11.461099 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:11.461325 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:11.461599 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 02:47:11.461780 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 02:47:11.466277 324 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:11.465180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60002 (local address 127.21.236.129:37257)
0811 02:47:11.465316 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:11.465320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:11.465528 (+ 208us) server_negotiation.cc:407] Connection header received
0811 02:47:11.465651 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:11.465654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:11.465689 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:11.465756 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:11.466187 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60002: BlockingRecv error: recv got EOF from 127.0.0.1:60002 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 02:47:11.466369 22450 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/krb5cc' of type FILE
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[31918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:11.467450 22450 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[31918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:11.468330 22450 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[31918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.129@KRBTEST.COM
I20260811 02:47:11.476732 324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:11.469160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.21.236.129:37257)
0811 02:47:11.469283 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:11.469287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:11.469411 (+ 124us) server_negotiation.cc:407] Connection header received
0811 02:47:11.469550 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:11.469553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:11.469585 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:11.469670 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:11.470160 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:11.470651 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:11.471513 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:11.471734 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:11.471839 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:11.473960 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:11.473967 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:11.473970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:11.473996 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:11.475362 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:11.475788 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:11.475791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:11.475792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:11.475864 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:11.476095 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:11.476098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:11.476099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:11.476250 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:11.476359 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:11.476538 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 02:47:11.476628 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 02:47:11.479321 324 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:11.478316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60028 (local address 127.21.236.129:37257)
0811 02:47:11.478453 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:11.478456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:11.478501 (+ 45us) server_negotiation.cc:407] Connection header received
0811 02:47:11.478605 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:11.478607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:11.478639 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:11.478702 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:11.479215 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60028: BlockingRecv error: recv got EOF from 127.0.0.1:60028 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 02:47:11.479373 22450 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786416395682572-22450-0/krb5kdc/krb5cc' of type FILE
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[31918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:11.480325 22450 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[31918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:11.481171 22450 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[31918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.129@KRBTEST.COM
I20260811 02:47:11.489650 324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:11.481967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60038 (local address 127.21.236.129:37257)
0811 02:47:11.482087 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:11.482090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:11.482141 (+ 51us) server_negotiation.cc:407] Connection header received
0811 02:47:11.482241 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:11.482243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:11.482273 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:11.482336 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:11.482774 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:11.483312 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:11.484057 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:11.484258 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:11.484339 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:11.486515 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:11.486518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:11.486521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:11.486546 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:11.487910 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:11.488384 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:11.488388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:11.488390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:11.488475 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:11.488775 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:11.488780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:11.488782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:11.488950 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:11.489137 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:11.489374 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 02:47:11.489488 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:47:11.490676 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 32115
I20260811 02:47:11.505232 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 32338
I20260811 02:47:11.522339 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 32575
I20260811 02:47:11.548856 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 31946
2026-08-11T02:47:11Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2588 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[344](info): set up 2 sockets
Aug 11 02:47:11 dist-test-slave-n326 krb5kdc[344](info): commencing operation
krb5kdc: starting...
W20260811 02:47:12.846833 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:47:12 dist-test-slave-n326 krb5kdc[344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:47:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:12Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.190.keytab.
Entry for principal oryx/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.190.keytab.
I20260811 02:47:12.982877 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:41631
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.190.keytab
--principal=oryx/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:41631
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:13.097141 366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:13.097486 366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:13.097574 366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:13.102300 366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:13.102392 366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:13.102430 366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:13.102487 366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:13.102550 366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:47:13.108151 366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:41631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.190.keytab
--principal=oryx/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:41631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:13.109301 366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:13.110383 366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:13.119230 380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:13.119231 375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:13.119230 376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:13.119357 366 server_base.cc:1034] running on GCE node
I20260811 02:47:13.120460 366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:13.121616 366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:13.122830 366 hybrid_clock.cc:648] HybridClock initialized: now 1786416433122790 us; error 55 us; skew 500 ppm
Aug 11 02:47:13 dist-test-slave-n326 krb5kdc[344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416433, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:13.125881 366 init.cc:377] Logged in from keytab as oryx/127.21.236.190@KRBTEST.COM (short username oryx)
I20260811 02:47:13.126763 366 webserver.cc:460] Webserver started at http://127.21.236.190:36357/ using document root <none> and password file <none>
I20260811 02:47:13.127403 366 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:13.127496 366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:13.127764 366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:13.129842 366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "2070708acfb041e58e41726a399e81be"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "41707b25dd39bda72d381ae747b1278e"
server_key_iv: "d53e990d57919339b90da168f4011940"
server_key_version: "encryptionkey@0"
I20260811 02:47:13.130407 366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "2070708acfb041e58e41726a399e81be"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "41707b25dd39bda72d381ae747b1278e"
server_key_iv: "d53e990d57919339b90da168f4011940"
server_key_version: "encryptionkey@0"
I20260811 02:47:13.134109 366 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 02:47:13.136808 392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:13.138274 366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:47:13.138451 366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "2070708acfb041e58e41726a399e81be"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "41707b25dd39bda72d381ae747b1278e"
server_key_iv: "d53e990d57919339b90da168f4011940"
server_key_version: "encryptionkey@0"
I20260811 02:47:13.138592 366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:13.163571 366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:13.167181 366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:13.167392 366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:13.174703 366 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:41631
I20260811 02:47:13.175135 474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:41631 every 8 connection(s)
I20260811 02:47:13.175761 366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:13.178617 481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:13.180697 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 366
I20260811 02:47:13.180840 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:47:13.181140 22450 external_mini_cluster.cc:1428] Setting key 6b5a510ff713978d071230cd6d9b0da4
I20260811 02:47:13.185389 481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be: Bootstrap starting.
I20260811 02:47:13.187685 481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:13.188567 481 log.cc:824] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:47:13 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
I20260811 02:47:13.190861 481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be: No bootstrap required, opened a new log
I20260811 02:47:13.194082 481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } }
I20260811 02:47:13.194309 481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:13.194401 481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2070708acfb041e58e41726a399e81be, State: Initialized, Role: FOLLOWER
I20260811 02:47:13.194825 481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } }
I20260811 02:47:13.194993 481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:13.195097 481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:13.195230 481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:13.196206 481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } }
I20260811 02:47:13.196545 481 leader_election.cc:302] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2070708acfb041e58e41726a399e81be; no voters:
I20260811 02:47:13.196892 481 leader_election.cc:288] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:47:13.196985 488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:13.197093 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:13.182428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60276 (local address 127.21.236.190:41631)
0811 02:47:13.182801 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:13.182816 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:13.182850 (+ 34us) server_negotiation.cc:407] Connection header received
0811 02:47:13.183541 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:13.183559 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:13.183882 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:13.184210 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:13.185033 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:13.186036 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:13.186880 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:13.187425 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:13.187598 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:13.190831 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:13.190836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:13.190849 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:13.190886 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:13.193484 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:13.194012 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:13.194027 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:13.194032 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:13.194155 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:13.194593 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:13.194597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:13.194599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:13.195207 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:13.195449 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:13.195794 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 02:47:13.196118 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":104,"threads_started":1}
I20260811 02:47:13.197216 488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [term 1 LEADER]: Becoming Leader. State: Replica: 2070708acfb041e58e41726a399e81be, State: Running, Role: LEADER
I20260811 02:47:13.197692 488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } }
I20260811 02:47:13.199391 481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:13.199352 489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } }
I20260811 02:47:13.199525 489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:13.199580 490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2070708acfb041e58e41726a399e81be. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } }
I20260811 02:47:13.199651 490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:13.200227 493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:13.203649 493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:13.210932 493 catalog_manager.cc:1347] Generated new cluster ID: fe3205df127e4363a58f17badb54d6dd
I20260811 02:47:13.211026 493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:13.258700 493 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:47:13.259814 493 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:13.269569 493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be: Generated new TSK 0
I20260811 02:47:13.270319 493 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.129.keytab.
Entry for principal oryx/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.129.keytab.
I20260811 02:47:13.319231 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:41631
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.129.keytab
--principal=oryx/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:13.435002 519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:13.435321 519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:13.435369 519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:13.439632 519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:13.439704 519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:13.439808 519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:47:13.445659 519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.129.keytab
--principal=oryx/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:41631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:13.446854 519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:13.447978 519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:13.458295 531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:13.458329 528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:13.458254 529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:13.458660 519 server_base.cc:1034] running on GCE node
I20260811 02:47:13.459213 519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:13.459890 519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:13.461102 519 hybrid_clock.cc:648] HybridClock initialized: now 1786416433461081 us; error 31 us; skew 500 ppm
Aug 11 02:47:13 dist-test-slave-n326 krb5kdc[344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416433, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:13.464275 519 init.cc:377] Logged in from keytab as oryx/127.21.236.129@KRBTEST.COM (short username oryx)
I20260811 02:47:13.465268 519 webserver.cc:460] Webserver started at http://127.21.236.129:37229/ using document root <none> and password file <none>
I20260811 02:47:13.465885 519 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:13.465984 519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:13.466240 519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:13.468489 519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "1e0426b9d6594109ba2a0afbdf86107d"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "a6aaadc7866c0bcd4738aca65580b6cd"
server_key_iv: "261a1339a7fde23308d7b5cdc10d795a"
server_key_version: "encryptionkey@0"
I20260811 02:47:13.469100 519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "1e0426b9d6594109ba2a0afbdf86107d"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "a6aaadc7866c0bcd4738aca65580b6cd"
server_key_iv: "261a1339a7fde23308d7b5cdc10d795a"
server_key_version: "encryptionkey@0"
I20260811 02:47:13.473182 519 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:47:13.476011 543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:13.477449 519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:47:13.477624 519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "1e0426b9d6594109ba2a0afbdf86107d"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "a6aaadc7866c0bcd4738aca65580b6cd"
server_key_iv: "261a1339a7fde23308d7b5cdc10d795a"
server_key_version: "encryptionkey@0"
I20260811 02:47:13.477767 519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:13.504819 519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:13.508584 519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:13.508826 519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:13.509510 519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:13.510478 519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:13.510555 519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:13.510630 519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:13.510674 519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:13.520586 519 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:42169
I20260811 02:47:13.520612 717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:42169 every 8 connection(s)
I20260811 02:47:13.521591 519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:47:13.527382 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 519
I20260811 02:47:13.527541 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:47:13.527917 22450 external_mini_cluster.cc:1428] Setting key 8c8087edac4621e76d12868c7faa9ce7
Aug 11 02:47:13 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416433, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
I20260811 02:47:13.538576 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:13.523703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:53067 (local address 127.21.236.190:41631)
0811 02:47:13.523896 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:13.523900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:13.524676 (+ 776us) server_negotiation.cc:407] Connection header received
0811 02:47:13.526131 (+ 1455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:13.526135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:13.526196 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:13.526299 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:13.527804 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:13.528572 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:13.530212 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:13.530507 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:13.530609 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:13.533845 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:13.533851 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:13.533858 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:13.533896 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:13.535935 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:13.536566 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:13.536573 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:13.536578 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:13.536689 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:13.537125 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:13.537132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:13.537135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:13.537414 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:13.537533 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:13.538229 (+ 696us) server_negotiation.cc:299] Negotiation successful
0811 02:47:13.538381 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
WARNING: no policy specified for oryx/127.21.236.130@KRBTEST.COM; defaulting to no policy
I20260811 02:47:13.539631 723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:13.524042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.129:53067)
0811 02:47:13.524515 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:13.524560 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:13.525915 (+ 1355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:13.526517 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:13.526524 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:13.526904 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:13.527631 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:13.527654 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:13.528712 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:13.528716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:13.530049 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:13.530064 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:13.530682 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:13.530689 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:13.530940 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:13.531751 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:13.531768 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:13.533638 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:13.536107 (+ 2469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:13.536112 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:13.536127 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:13.536450 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:13.536797 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:13.536800 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:13.536801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:13.537018 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:13.537589 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:13.537596 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:13.537916 (+ 320us) client_negotiation.cc:769] Sending connection context
0811 02:47:13.538139 (+ 223us) client_negotiation.cc:240] Negotiation successful
0811 02:47:13.538470 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":122,"threads_started":1}
I20260811 02:47:13.541112 721 heartbeater.cc:347] Connected to a master server at 127.21.236.190:41631
I20260811 02:47:13.541458 721 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:13.542119 721 heartbeater.cc:511] Master 127.21.236.190:41631 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.236.130@KRBTEST.COM" created.
I20260811 02:47:13.544075 417 ts_manager.cc:194] Registered new tserver with Master: 1e0426b9d6594109ba2a0afbdf86107d (127.21.236.129:42169)
I20260811 02:47:13.545321 417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:53067
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.130.keytab.
Entry for principal oryx/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.130.keytab.
I20260811 02:47:13.563536 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:41631
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.130.keytab
--principal=oryx/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:13.685801 726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:13.686162 726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:13.686259 726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:13.690482 726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:13.690591 726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:13.690693 726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:47:13.696285 726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.130.keytab
--principal=oryx/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:41631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:13.697482 726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:13.698580 726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:13.709520 734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:13.709522 737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:13.709520 742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:13.710006 726 server_base.cc:1034] running on GCE node
I20260811 02:47:13.710587 726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:13.711676 726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:13.712911 726 hybrid_clock.cc:648] HybridClock initialized: now 1786416433712847 us; error 49 us; skew 500 ppm
Aug 11 02:47:13 dist-test-slave-n326 krb5kdc[344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416433, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:13.716254 726 init.cc:377] Logged in from keytab as oryx/127.21.236.130@KRBTEST.COM (short username oryx)
I20260811 02:47:13.717295 726 webserver.cc:460] Webserver started at http://127.21.236.130:34761/ using document root <none> and password file <none>
I20260811 02:47:13.717971 726 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:13.718091 726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:13.718361 726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:13.720825 726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "0b645f57220a478cb25806d0c7da19e8"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "d9dc7bc9b6a812b924976d19c58181e0"
server_key_iv: "39b2b333cfdda2c28e8cf994128ba1f4"
server_key_version: "encryptionkey@0"
I20260811 02:47:13.721540 726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "0b645f57220a478cb25806d0c7da19e8"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "d9dc7bc9b6a812b924976d19c58181e0"
server_key_iv: "39b2b333cfdda2c28e8cf994128ba1f4"
server_key_version: "encryptionkey@0"
I20260811 02:47:13.726207 726 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 02:47:13.729156 757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:13.730682 726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:47:13.730852 726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "0b645f57220a478cb25806d0c7da19e8"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "d9dc7bc9b6a812b924976d19c58181e0"
server_key_iv: "39b2b333cfdda2c28e8cf994128ba1f4"
server_key_version: "encryptionkey@0"
I20260811 02:47:13.730984 726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:13.749729 726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:13.753712 726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:13.753965 726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:13.754674 726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:13.755662 726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:13.755743 726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:13.755824 726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:13.755874 726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:13.765893 726 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:33221
I20260811 02:47:13.765946 943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:33221 every 8 connection(s)
I20260811 02:47:13.766892 726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:47:13.771936 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 726
I20260811 02:47:13.772099 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:47:13.772457 22450 external_mini_cluster.cc:1428] Setting key f3f651e39c8238930ebd4733efababca
Aug 11 02:47:13 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416433, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.21.236.131@KRBTEST.COM; defaulting to no policy
I20260811 02:47:13.784118 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:13.769075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:48337 (local address 127.21.236.190:41631)
0811 02:47:13.769224 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:13.769228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:13.770210 (+ 982us) server_negotiation.cc:407] Connection header received
0811 02:47:13.771735 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:13.771742 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:13.771809 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:13.771953 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:13.773477 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:13.774182 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:13.776201 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:13.776497 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:13.776654 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:13.779731 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:13.779734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:13.779737 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:13.779763 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:13.781453 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:13.782185 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:13.782188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:13.782190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:13.782274 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:13.782678 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:13.782682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:13.782684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:13.783035 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:13.783181 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:13.783767 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 02:47:13.783912 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:47:13.785028 946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:13.769553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.130:48337)
0811 02:47:13.770042 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:13.770090 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:13.771482 (+ 1392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:13.772176 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:13.772186 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:13.772688 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:13.773328 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:13.773344 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:13.774350 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:13.774355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:13.776047 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:13.776064 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:13.776706 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:13.776712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:13.776927 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:13.777663 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:13.777683 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:13.779527 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:13.781596 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:13.781603 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:13.781679 (+ 76us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:13.782052 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:13.782388 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:13.782391 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:13.782393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:13.782561 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:13.783178 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:13.783187 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:13.783488 (+ 301us) client_negotiation.cc:769] Sending connection context
0811 02:47:13.783703 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 02:47:13.784085 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":124,"threads_started":1}
I20260811 02:47:13.786526 944 heartbeater.cc:347] Connected to a master server at 127.21.236.190:41631
I20260811 02:47:13.786814 944 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.236.131@KRBTEST.COM" created.
I20260811 02:47:13.787505 944 heartbeater.cc:511] Master 127.21.236.190:41631 requested a full tablet report, sending...
I20260811 02:47:13.788838 417 ts_manager.cc:194] Registered new tserver with Master: 0b645f57220a478cb25806d0c7da19e8 (127.21.236.130:33221)
I20260811 02:47:13.789716 417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:48337
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.131.keytab.
Entry for principal oryx/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.131.keytab.
I20260811 02:47:13.806380 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:41631
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.131.keytab
--principal=oryx/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:13.923262 949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:13.923542 949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:13.923586 949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:13.927507 949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:13.927564 949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:13.927623 949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:47:13.932832 949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.131.keytab
--principal=oryx/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:41631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:13.933856 949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:13.934980 949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:13.945241 955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:13.945253 958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:13.945253 954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:13.945343 949 server_base.cc:1034] running on GCE node
I20260811 02:47:13.945989 949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:13.946693 949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:13.947918 949 hybrid_clock.cc:648] HybridClock initialized: now 1786416433947884 us; error 52 us; skew 500 ppm
Aug 11 02:47:13 dist-test-slave-n326 krb5kdc[344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416433, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:13.950906 949 init.cc:377] Logged in from keytab as oryx/127.21.236.131@KRBTEST.COM (short username oryx)
I20260811 02:47:13.951877 949 webserver.cc:460] Webserver started at http://127.21.236.131:44991/ using document root <none> and password file <none>
I20260811 02:47:13.952462 949 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:13.952600 949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:13.952854 949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:13.954986 949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "70311aa80d1841af979f44135d73f088"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "79b557928b79e0c63eb21bf2585d741b"
server_key_iv: "f24ad2e3331b9db8deb94ba4e9b19286"
server_key_version: "encryptionkey@0"
I20260811 02:47:13.955659 949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "70311aa80d1841af979f44135d73f088"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "79b557928b79e0c63eb21bf2585d741b"
server_key_iv: "f24ad2e3331b9db8deb94ba4e9b19286"
server_key_version: "encryptionkey@0"
I20260811 02:47:13.959440 949 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:47:13.961918 969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:13.963238 949 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:47:13.963388 949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "70311aa80d1841af979f44135d73f088"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "79b557928b79e0c63eb21bf2585d741b"
server_key_iv: "f24ad2e3331b9db8deb94ba4e9b19286"
server_key_version: "encryptionkey@0"
I20260811 02:47:13.963506 949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:13.977205 949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:13.980329 949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:13.980486 949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:13.981015 949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:13.981889 949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:13.981942 949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:13.981982 949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:13.981998 949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:13.990913 949 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:46517
I20260811 02:47:13.990991 1164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:46517 every 8 connection(s)
I20260811 02:47:13.991874 949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:47:13.994868 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 949
I20260811 02:47:13.995018 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:47:13.995368 22450 external_mini_cluster.cc:1428] Setting key 539f7db8a153caec149831d872775e31
Aug 11 02:47:13 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416433, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
I20260811 02:47:14.005848 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:13.993788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:43359 (local address 127.21.236.190:41631)
0811 02:47:13.993936 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:13.993941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:13.994745 (+ 804us) server_negotiation.cc:407] Connection header received
0811 02:47:13.996469 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:13.996472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:13.996528 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:13.996605 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:13.997724 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:13.998370 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:13.999530 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:13.999757 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:13.999885 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.002477 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.002480 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.002482 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.002508 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.003873 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.004339 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.004341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.004343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.004415 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.004739 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.004762 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.004766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.004993 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.005076 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.005615 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.005723 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:47:14.006704 1167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:13.994168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.131:43359)
0811 02:47:13.994598 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:13.994638 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:13.996253 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:13.996770 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:13.996777 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:13.997114 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:13.997581 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:13.997595 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:13.998491 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:13.998495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:13.999396 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:13.999406 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:13.999921 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:13.999926 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.000098 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.000623 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.000639 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.002307 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.003977 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.003982 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.003993 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.004254 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.004493 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.004495 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.004496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.004635 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.005102 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.005108 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.005369 (+ 261us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.005532 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.005791 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":106,"threads_started":1}
I20260811 02:47:14.008010 1165 heartbeater.cc:347] Connected to a master server at 127.21.236.190:41631
I20260811 02:47:14.008281 1165 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:14.008834 1165 heartbeater.cc:511] Master 127.21.236.190:41631 requested a full tablet report, sending...
I20260811 02:47:14.009828 417 ts_manager.cc:194] Registered new tserver with Master: 70311aa80d1841af979f44135d73f088 (127.21.236.131:46517)
I20260811 02:47:14.010411 417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:43359
I20260811 02:47:14.020992 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.236.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.236.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.236.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.189.keytab.
Entry for principal oryx/127.21.236.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.189.keytab.
I20260811 02:47:14.052557 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.189:46833
--webserver_interface=127.21.236.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.189.keytab
--principal=oryx/127.21.236.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:41631,127.21.236.189:46833
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:14.166870 1173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:14.167201 1173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:14.167273 1173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:14.171442 1173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:14.171502 1173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:14.171521 1173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:14.171538 1173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:14.171554 1173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:47:14.177843 1173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:41631,127.21.236.189:46833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.189.keytab
--principal=oryx/127.21.236.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.189:46833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.236.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
W20260811 02:47:14.178083 1173 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.21.236.190:41631,127.21.236.189:46833'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 02:47:14.179138 1173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:14.180364 1173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:14.190862 1187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:14.190904 1185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:14.190958 1189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:14.191278 1173 server_base.cc:1034] running on GCE node
I20260811 02:47:14.191829 1173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:14.192983 1173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:14.194245 1173 hybrid_clock.cc:648] HybridClock initialized: now 1786416434194230 us; error 28 us; skew 500 ppm
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416434, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:14.200701 1173 init.cc:377] Logged in from keytab as oryx/127.21.236.189@KRBTEST.COM (short username oryx)
I20260811 02:47:14.201916 1173 webserver.cc:460] Webserver started at http://127.21.236.189:46689/ using document root <none> and password file <none>
I20260811 02:47:14.202543 1173 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:14.202641 1173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:14.202865 1173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:14.205421 1173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/data/instance:
uuid: "e56f6424c0994b6f8ad2bbab85626f10"
format_stamp: "Formatted at 2026-08-11 02:47:14 on dist-test-slave-n326"
server_key: "f1e590b4a9f82fbc92299b9e7c9447ef"
server_key_iv: "fe5fa066ce4f84ad869bdee779c3bd70"
server_key_version: "encryptionkey@0"
I20260811 02:47:14.206173 1173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/wal/instance:
uuid: "e56f6424c0994b6f8ad2bbab85626f10"
format_stamp: "Formatted at 2026-08-11 02:47:14 on dist-test-slave-n326"
server_key: "f1e590b4a9f82fbc92299b9e7c9447ef"
server_key_iv: "fe5fa066ce4f84ad869bdee779c3bd70"
server_key_version: "encryptionkey@0"
I20260811 02:47:14.210989 1173 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 02:47:14.213519 1200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:14.214799 1173 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:47:14.214953 1173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/wal
uuid: "e56f6424c0994b6f8ad2bbab85626f10"
format_stamp: "Formatted at 2026-08-11 02:47:14 on dist-test-slave-n326"
server_key: "f1e590b4a9f82fbc92299b9e7c9447ef"
server_key_iv: "fe5fa066ce4f84ad869bdee779c3bd70"
server_key_version: "encryptionkey@0"
I20260811 02:47:14.215118 1173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:14.231204 1173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:14.234674 1173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:14.234881 1173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:14.242117 1173 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.189:46833
I20260811 02:47:14.242409 1281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.189:46833 every 8 connection(s)
I20260811 02:47:14.243399 1173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/data/info.pb
I20260811 02:47:14.246213 1283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:14.249653 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 1173
I20260811 02:47:14.250104 1283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416434, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.189@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.236.189@KRBTEST.COM
I20260811 02:47:14.262897 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.252134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60292 (local address 127.21.236.190:41631)
0811 02:47:14.252287 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.252293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.252630 (+ 337us) server_negotiation.cc:407] Connection header received
0811 02:47:14.252982 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.252986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.253044 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.253124 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.254169 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.254988 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.256117 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.256346 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.256426 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.259276 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.259280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.259282 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.259308 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.260940 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.261454 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.261458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.261460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.261546 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.261869 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.261874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.261875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.262111 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.262226 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.262598 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.262721 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 02:47:14.263768 1288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.252049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.0.0.1:60292)
0811 02:47:14.252444 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.252491 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.252814 (+ 323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.253202 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.253211 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.253673 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.254031 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.254044 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.255167 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.255173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.255982 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.255991 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.256485 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.256489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.256810 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.257506 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.257547 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.259022 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.261066 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.261077 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.261087 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.261333 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.261642 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.261645 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.261647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.261768 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.262223 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.262229 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.262374 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.262558 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.262882 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":78,"spinlock_wait_cycles":2176,"thread_start_us":143,"threads_started":1}
I20260811 02:47:14.263772 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.251217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37654 (local address 127.21.236.189:46833)
0811 02:47:14.251634 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.251651 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.251691 (+ 40us) server_negotiation.cc:407] Connection header received
0811 02:47:14.252495 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.252516 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.252808 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.253040 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.253979 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.255277 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.256143 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.256563 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.256806 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.259603 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.259608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.259618 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.259638 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.261135 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.261594 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.261604 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.261605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.261704 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.261998 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.262001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.262003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.262355 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.262583 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.262753 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.263094 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":151,"server-negotiator.queue_time_us":270,"spinlock_wait_cycles":768,"thread_start_us":99,"threads_started":1}
I20260811 02:47:14.265242 1283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416434, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.189@KRBTEST.COM for oryx/127.21.236.189@KRBTEST.COM
I20260811 02:47:14.273751 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.265785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.21.236.189:46833)
0811 02:47:14.265885 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.265892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.265939 (+ 47us) server_negotiation.cc:407] Connection header received
0811 02:47:14.266087 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.266090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.266132 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.266212 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.266584 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.267552 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.268427 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.268663 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.268778 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.270720 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.270723 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.270725 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.270743 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.272186 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.272604 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.272610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.272612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.272687 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.272966 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.272969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.272971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.273138 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.273242 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.273477 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.273627 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":37}
I20260811 02:47:14.273751 1288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.265729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:46833 (local address 127.0.0.1:37662)
0811 02:47:14.265882 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.265896 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.265977 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.266222 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.266225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.266390 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.266471 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.266476 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.267661 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.267667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.268308 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.268317 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.268770 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.268774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.268870 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.269348 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.269361 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.270577 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.272313 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.272318 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.272319 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.272509 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.272781 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.272787 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.272789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.272872 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.273230 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.273233 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.273314 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.273432 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.273570 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 02:47:14.276667 1283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: Bootstrap starting.
I20260811 02:47:14.278955 1283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:14.279732 1283 log.cc:824] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:14.281778 1283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: No bootstrap required, opened a new log
I20260811 02:47:14.284837 1283 raft_consensus.cc:348] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } }
I20260811 02:47:14.285060 1283 raft_consensus.cc:374] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:14.285142 1283 raft_consensus.cc:729] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e56f6424c0994b6f8ad2bbab85626f10, State: Initialized, Role: FOLLOWER
I20260811 02:47:14.285573 1283 consensus_queue.cc:260] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } }
I20260811 02:47:14.286052 1293 sys_catalog.cc:455] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } } }
I20260811 02:47:14.286164 1293 sys_catalog.cc:458] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:47:14.286747 1283 sys_catalog.cc:564] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:47:14.291560 1310 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:47:14.291652 1310 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:47:14.293854 1173 master_runner.cc:428] Detected that this master e56f6424c0994b6f8ad2bbab85626f10 is joining an existing cluster
I20260811 02:47:14.293924 1173 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.236.189:46833
I20260811 02:47:14.304723 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.296734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37668 (local address 127.21.236.189:46833)
0811 02:47:14.296821 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.296826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.297218 (+ 392us) server_negotiation.cc:407] Connection header received
0811 02:47:14.297306 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.297309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.297353 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.297432 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.297970 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.298605 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.299506 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.299826 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.299925 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.301290 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.301294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.301297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.301318 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.303021 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.303488 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.303494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.303496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.303591 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.303861 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.303867 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.303869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.304082 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.304172 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.304419 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.304583 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":30,"spinlock_wait_cycles":2432}
I20260811 02:47:14.304736 1323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.296754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:46833 (local address 127.0.0.1:37668)
0811 02:47:14.297114 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.297129 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.297234 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.297469 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.297472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.297706 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.297846 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.297854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.298720 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.298724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.299396 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.299406 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.299959 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.299963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.300056 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.300671 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.300692 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.301156 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.303188 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.303191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.303193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.303389 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.303687 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.303690 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.303692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.303776 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.304158 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.304161 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.304265 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.304374 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.304522 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":21,"thread_start_us":92,"threads_started":1}
I20260811 02:47:14.305241 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.296308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60304 (local address 127.21.236.190:41631)
0811 02:47:14.296467 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.296472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.296619 (+ 147us) server_negotiation.cc:407] Connection header received
0811 02:47:14.296783 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.296788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.296841 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.296899 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.297621 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.298306 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.299632 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.299967 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.300063 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.301425 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.301429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.301432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.301462 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.303412 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.303926 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.303929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.303931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.304010 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.304326 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.304330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.304332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.304555 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.304627 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.304964 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.305085 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 02:47:14.305842 1320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.296240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.0.0.1:60304)
0811 02:47:14.296520 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.296537 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.296650 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.297084 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.297086 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.297347 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.297499 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.297506 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.298423 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.298426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.299445 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.299459 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.300081 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.300085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.300177 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.300827 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.300841 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.301256 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.303551 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.303554 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.303557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.303799 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.304120 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.304123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.304125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.304214 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.304700 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.304703 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.304786 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.305454 (+ 668us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.305649 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":24,"thread_start_us":102,"threads_started":1}
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416433, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.190@KRBTEST.COM for oryx/127.21.236.189@KRBTEST.COM
I20260811 02:47:14.320802 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.310737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37672 (local address 127.21.236.189:46833)
0811 02:47:14.310864 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.310869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.311260 (+ 391us) server_negotiation.cc:407] Connection header received
0811 02:47:14.311454 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.311457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.311508 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.311599 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.312290 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.313053 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.313953 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.314286 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.314387 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.316742 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.316746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.316749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.316777 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.318561 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.319369 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.319373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.319375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.319466 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.319783 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.319789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.319793 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.320049 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.320157 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.320526 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.320643 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:47:14.320839 1324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.310679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:46833 (local address 127.0.0.1:37672)
0811 02:47:14.311127 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.311182 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.311317 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.311688 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.311694 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.312050 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.312174 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.312181 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.313168 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.313171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.313829 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.313840 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.314395 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.314398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.314485 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.315042 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.315054 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.316605 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.318698 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.318711 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.318722 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.319250 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.319571 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.319577 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.319582 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.319679 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.320176 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.320184 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.320305 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.320457 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.320656 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":112,"threads_started":1}
I20260811 02:47:14.322607 417 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.236.189:46833: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10"
member_type: NON_VOTER
last_known_addr {
host: "127.21.236.189"
port: 46833
}
attrs {
promote: true
}
}
dest_uuid: "2070708acfb041e58e41726a399e81be"
cas_config_opid_index: -1
I20260811 02:47:14.323426 417 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } }
I20260811 02:47:14.325738 1330 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e56f6424c0994b6f8ad2bbab85626f10 (127.21.236.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } }
I20260811 02:47:14.326666 1239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:14.327867 1330 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.236.189:46833
I20260811 02:47:14.328331 1239 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 FOLLOWER]: Refusing update from remote peer 2070708acfb041e58e41726a399e81be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 02:47:14.328369 1173 master.cc:537] Master@127.21.236.189:46833 shutting down...
WARNING: no policy specified for oryx/127.21.236.188@KRBTEST.COM; defaulting to no policy
I20260811 02:47:14.329270 1335 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:14.329790 1331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } }
I20260811 02:47:14.329897 1331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.236.188@KRBTEST.COM" created.
I20260811 02:47:14.333395 1173 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:14.333709 1173 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:14.333837 1173 tablet_replica.cc:330] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: stopping tablet replica
W20260811 02:47:14.335232 396 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 02:47:14.336678 396 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be -> Peer e56f6424c0994b6f8ad2bbab85626f10 (127.21.236.189:46833): Couldn't send request to peer e56f6424c0994b6f8ad2bbab85626f10. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 02:47:14.338665 1173 master.cc:552] Master@127.21.236.189:46833 shutdown complete.
I20260811 02:47:14.338737 1173 master_runner.cc:305] Clearing existing system tablet
I20260811 02:47:14.338966 1173 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:14.339041 1173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:14.340139 1173 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 02:47:14.340999 1342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:14.341202 1173 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:14.341346 1173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 02:47:14.341401 1173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/wal
uuid: "e56f6424c0994b6f8ad2bbab85626f10"
format_stamp: "Formatted at 2026-08-11 02:47:14 on dist-test-slave-n326"
server_key: "f1e590b4a9f82fbc92299b9e7c9447ef"
server_key_iv: "fe5fa066ce4f84ad869bdee779c3bd70"
server_key_version: "encryptionkey@0"
I20260811 02:47:14.343219 1173 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:14.345208 1173 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 02:47:14.345286 1173 log.cc:1196] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 02:47:14.345558 1173 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: Deleting consensus metadata
I20260811 02:47:14.345736 1173 master_runner.cc:315] Copying system tablet from 127.21.236.190:41631
I20260811 02:47:14.346932 1173 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: tablet copy: Beginning tablet copy session from remote peer at address 127.21.236.190:41631
I20260811 02:47:14.356230 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.347601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60320 (local address 127.21.236.190:41631)
0811 02:47:14.347732 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.347738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.347941 (+ 203us) server_negotiation.cc:407] Connection header received
0811 02:47:14.348099 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.348102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.348156 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.348240 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.348679 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.349243 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.350104 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.350553 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.350698 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.352161 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.352165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.352168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.352196 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.354105 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.354644 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.354648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.354650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.354740 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.355057 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.355163 (+ 106us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.355165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.355469 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.355554 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.355920 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.356026 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:47:14.356237 1351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.347535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.0.0.1:60320)
0811 02:47:14.347848 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.347860 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.347991 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.348256 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.348259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.348474 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.348558 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.348564 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.349385 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.349388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.349980 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.349988 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.350711 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.350716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.350831 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.351512 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.351528 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.351996 (+ 468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.354227 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.354230 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.354233 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.354527 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.354840 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.354846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.354848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.354949 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.355577 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.355581 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.355694 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.355819 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.355971 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":94,"threads_started":1}
I20260811 02:47:14.356916 448 tablet_copy_service.cc:140] P 2070708acfb041e58e41726a399e81be: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e56f6424c0994b6f8ad2bbab85626f10 ({username='oryx', principal='oryx/127.21.236.189@KRBTEST.COM'} at 127.0.0.1:60320)
I20260811 02:47:14.357149 448 tablet_copy_service.cc:161] P 2070708acfb041e58e41726a399e81be: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e56f6424c0994b6f8ad2bbab85626f10 at {username='oryx', principal='oryx/127.21.236.189@KRBTEST.COM'} at 127.0.0.1:60320: session id = e56f6424c0994b6f8ad2bbab85626f10-00000000000000000000000000000000
I20260811 02:47:14.358812 448 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 02:47:14.360561 1173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.236.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.188.keytab.
Entry for principal oryx/127.21.236.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.188.keytab.
I20260811 02:47:14.362819 1173 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: tablet copy: Starting download of 0 data blocks...
I20260811 02:47:14.362990 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.188:34191
--webserver_interface=127.21.236.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.188.keytab
--principal=oryx/127.21.236.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:41631,127.21.236.189:46833,127.21.236.188:34191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:47:14.363044 1173 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: tablet copy: Starting download of 1 WAL segments...
I20260811 02:47:14.364827 1173 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 02:47:14.366858 448 tablet_copy_service.cc:342] P 2070708acfb041e58e41726a399e81be: Request end of tablet copy session e56f6424c0994b6f8ad2bbab85626f10-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.236.189@KRBTEST.COM'} at 127.0.0.1:60320
I20260811 02:47:14.367003 448 tablet_copy_service.cc:433] P 2070708acfb041e58e41726a399e81be: ending tablet copy session e56f6424c0994b6f8ad2bbab85626f10-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e56f6424c0994b6f8ad2bbab85626f10
I20260811 02:47:14.383976 1173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:14.397773 1358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:14.397795 1354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:14.398134 1355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:14.398605 1173 server_base.cc:1034] running on GCE node
I20260811 02:47:14.398903 1173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:14.399269 1173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:14.406610 1173 hybrid_clock.cc:648] HybridClock initialized: now 1786416434406585 us; error 41 us; skew 500 ppm
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416434, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:14.410921 1173 init.cc:377] Logged in from keytab as oryx/127.21.236.189@KRBTEST.COM (short username oryx)
I20260811 02:47:14.415649 1173 webserver.cc:460] Webserver started at http://127.21.236.189:44399/ using document root <none> and password file <none>
I20260811 02:47:14.415876 1173 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:14.415913 1173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:14.417363 1173 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 02:47:14.418586 1371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:14.418869 1173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 02:47:14.418998 1173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/wal
uuid: "e56f6424c0994b6f8ad2bbab85626f10"
format_stamp: "Formatted at 2026-08-11 02:47:14 on dist-test-slave-n326"
server_key: "f1e590b4a9f82fbc92299b9e7c9447ef"
server_key_iv: "fe5fa066ce4f84ad869bdee779c3bd70"
server_key_version: "encryptionkey@0"
I20260811 02:47:14.419185 1173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:14.435695 1173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:14.438048 1173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:14.443706 1173 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.189:46833
I20260811 02:47:14.444389 1173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-1/data/info.pb
I20260811 02:47:14.446009 1456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.189:46833 every 8 connection(s)
I20260811 02:47:14.446353 1457 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:47:14.447425 1457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: Bootstrap starting.
I20260811 02:47:14.458623 1457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:47:14.459026 1457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: Bootstrap complete.
I20260811 02:47:14.459810 1457 raft_consensus.cc:348] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } }
I20260811 02:47:14.459996 1457 raft_consensus.cc:729] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56f6424c0994b6f8ad2bbab85626f10, State: Initialized, Role: LEARNER
I20260811 02:47:14.460117 1457 consensus_queue.cc:260] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } }
I20260811 02:47:14.460511 1461 sys_catalog.cc:455] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } }
I20260811 02:47:14.460690 1457 sys_catalog.cc:564] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:14.460709 1461 sys_catalog.cc:458] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:47:14.464639 1482 catalog_manager.cc:1259] Loaded cluster ID: fe3205df127e4363a58f17badb54d6dd
I20260811 02:47:14.464708 1482 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: loading cluster ID for follower catalog manager: success
I20260811 02:47:14.466817 1482 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: acquiring CA information for follower catalog manager: success
I20260811 02:47:14.468281 1482 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 02:47:14.527406 1352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:14.527660 1352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:14.527701 1352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:14.531591 1352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:14.531644 1352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:14.531664 1352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:14.531680 1352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:14.531697 1352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:47:14.537137 1352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:41631,127.21.236.189:46833,127.21.236.188:34191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.188.keytab
--principal=oryx/127.21.236.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.188:34191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.236.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:14.538208 1352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:14.539209 1352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:14.548127 1487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:14.548092 721 heartbeater.cc:503] Master 127.21.236.190:41631 was elected leader, sending a full tablet report...
W20260811 02:47:14.548032 1493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:14.548022 1488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:14.548323 1352 server_base.cc:1034] running on GCE node
I20260811 02:47:14.549218 1352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:14.550330 1352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:14.551541 1352 hybrid_clock.cc:648] HybridClock initialized: now 1786416434551494 us; error 52 us; skew 500 ppm
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416434, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:14.554427 1352 init.cc:377] Logged in from keytab as oryx/127.21.236.188@KRBTEST.COM (short username oryx)
I20260811 02:47:14.555384 1352 webserver.cc:460] Webserver started at http://127.21.236.188:36161/ using document root <none> and password file <none>
I20260811 02:47:14.556016 1352 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:14.556109 1352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:14.556348 1352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:14.558480 1352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/data/instance:
uuid: "f60bba59016c4a489122f38ed2bf24bf"
format_stamp: "Formatted at 2026-08-11 02:47:14 on dist-test-slave-n326"
server_key: "79a504765065fe739799c2e5844bebb4"
server_key_iv: "887dac1fe6145b7de5e049c89d8f0b29"
server_key_version: "encryptionkey@0"
I20260811 02:47:14.559055 1352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/wal/instance:
uuid: "f60bba59016c4a489122f38ed2bf24bf"
format_stamp: "Formatted at 2026-08-11 02:47:14 on dist-test-slave-n326"
server_key: "79a504765065fe739799c2e5844bebb4"
server_key_iv: "887dac1fe6145b7de5e049c89d8f0b29"
server_key_version: "encryptionkey@0"
I20260811 02:47:14.563195 1352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:47:14.565636 1506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:14.567180 1352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:47:14.567322 1352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/wal
uuid: "f60bba59016c4a489122f38ed2bf24bf"
format_stamp: "Formatted at 2026-08-11 02:47:14 on dist-test-slave-n326"
server_key: "79a504765065fe739799c2e5844bebb4"
server_key_iv: "887dac1fe6145b7de5e049c89d8f0b29"
server_key_version: "encryptionkey@0"
I20260811 02:47:14.567447 1352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:14.594098 1352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:14.597846 1352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:14.598068 1352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:14.607125 1352 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.188:34191
I20260811 02:47:14.607492 1593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.188:34191 every 8 connection(s)
I20260811 02:47:14.608312 1352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/data/info.pb
I20260811 02:47:14.611392 1594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:14.615162 1594 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:47:14.616897 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 1352
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416434, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.188@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.236.188@KRBTEST.COM
I20260811 02:47:14.630910 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.617413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60330 (local address 127.21.236.190:41631)
0811 02:47:14.617561 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.617565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.618521 (+ 956us) server_negotiation.cc:407] Connection header received
0811 02:47:14.619549 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.619562 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.619660 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.619753 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.620938 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.621878 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.623367 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.623760 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.623874 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.626878 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.626886 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.626888 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.626917 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.628556 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.629143 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.629147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.629149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.629233 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.629620 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.629623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.629624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.629915 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.630022 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.630579 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.630711 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:47:14.631762 1598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.617867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45046 (local address 127.21.236.188:34191)
0811 02:47:14.618392 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.618408 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.618445 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:47:14.619282 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.619300 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.619545 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.619793 (+ 248us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.620764 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.622183 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.623367 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.623996 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.624132 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.627508 (+ 3376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.627515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.627536 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.627558 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.629324 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.629790 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.629796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.629798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.629893 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.630201 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.630203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.630204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.630455 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.630699 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.630907 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.631324 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":437,"spinlock_wait_cycles":60288,"thread_start_us":197,"threads_started":1}
I20260811 02:47:14.631763 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.617794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.0.0.1:60330)
0811 02:47:14.618336 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.618395 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.619293 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.619822 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.619827 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.620311 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.620781 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.620800 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.622036 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.622038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.623245 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.623254 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.623921 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.623924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.624079 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.624879 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.624902 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.626727 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.628676 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.628684 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.628695 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.629037 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.629329 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.629332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.629334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.629513 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.630032 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.630039 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.630342 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.630495 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.630844 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":54,"spinlock_wait_cycles":2944,"thread_start_us":175,"threads_started":1}
I20260811 02:47:14.633000 1594 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416434, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.188@KRBTEST.COM for oryx/127.21.236.189@KRBTEST.COM
I20260811 02:47:14.641052 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.633513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:46833 (local address 127.0.0.1:37674)
0811 02:47:14.633669 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.633680 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.633768 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.634143 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.634145 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.634269 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.634422 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.634427 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.635418 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.635422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.635952 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.635960 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.636394 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.636397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.636469 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.636842 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.636853 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.638046 (+ 1193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.639830 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.639833 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.639834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.640006 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.640276 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.640278 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.640280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.640351 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.640701 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.640703 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.640776 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.640854 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.640946 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 02:47:14.641242 1603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.633639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37674 (local address 127.21.236.189:46833)
0811 02:47:14.633896 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.633899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.633912 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:47:14.633956 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.633958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.633998 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.634118 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.634527 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.635282 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.636066 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.636292 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.636381 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.638172 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.638175 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.638177 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.638200 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.639738 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.640115 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.640117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.640118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.640193 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.640453 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.640455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.640456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.640619 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.640695 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.640891 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.640986 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":104,"threads_started":1}
I20260811 02:47:14.642019 1594 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416434, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.188@KRBTEST.COM for oryx/127.21.236.188@KRBTEST.COM
I20260811 02:47:14.650287 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.642485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:34191 (local address 127.0.0.1:45050)
0811 02:47:14.642634 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.642648 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.642727 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.643024 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.643027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.643213 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.643275 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.643280 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.644007 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.644010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.644507 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.644515 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.644985 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.644988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.645111 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.645574 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.645613 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.646896 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.648840 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.648843 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.648845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.649046 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.649329 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.649332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.649334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.649422 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.649784 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.649788 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.649868 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.649980 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.650114 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":34}
I20260811 02:47:14.650287 1598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.642579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45050 (local address 127.21.236.188:34191)
0811 02:47:14.642689 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.642693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.642705 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:47:14.642888 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.642891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.642934 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.643015 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.643387 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.643883 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.644625 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.644878 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.644988 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.647025 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.647028 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.647031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.647053 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.648701 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.649145 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.649150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.649152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.649232 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.649510 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.649515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.649517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.649682 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.649784 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.650025 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.650166 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":42}
I20260811 02:47:14.653868 1594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: Bootstrap starting.
I20260811 02:47:14.656286 1594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:14.657035 1594 log.cc:824] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:14.659022 1594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: No bootstrap required, opened a new log
I20260811 02:47:14.662581 1594 raft_consensus.cc:348] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } }
I20260811 02:47:14.662799 1594 raft_consensus.cc:374] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:14.662833 1594 raft_consensus.cc:729] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f60bba59016c4a489122f38ed2bf24bf, State: Initialized, Role: FOLLOWER
I20260811 02:47:14.663390 1594 consensus_queue.cc:260] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } }
I20260811 02:47:14.663930 1606 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } } }
I20260811 02:47:14.664054 1606 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:47:14.664669 1594 sys_catalog.cc:564] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:47:14.669489 1624 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:47:14.669586 1624 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:47:14.673667 1352 master_runner.cc:428] Detected that this master f60bba59016c4a489122f38ed2bf24bf is joining an existing cluster
I20260811 02:47:14.673751 1352 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.236.188:34191
I20260811 02:47:14.685376 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 519
I20260811 02:47:14.690886 1598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.676180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45052 (local address 127.21.236.188:34191)
0811 02:47:14.676318 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.676323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.676585 (+ 262us) server_negotiation.cc:407] Connection header received
0811 02:47:14.676723 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.676726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.676764 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.676810 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.677643 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.678567 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.680518 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.680924 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.681022 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.682631 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.682635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.682638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.682662 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.684311 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.684904 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.684907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.684909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.685001 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.685289 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.685292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.685294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.685515 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.690184 (+ 4669us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.690522 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.690730 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":195,"server-negotiator.queue_time_us":51}
I20260811 02:47:14.690900 1634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.676148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:34191 (local address 127.0.0.1:45052)
0811 02:47:14.676498 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.676514 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.676620 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.677131 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.677134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.677378 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.677525 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.677533 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.678685 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.678688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.680381 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.680395 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.681147 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.681153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.681332 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.681995 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.682021 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.682495 (+ 474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.684518 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.684521 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.684524 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.684781 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.685103 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.685108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.685113 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.685201 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.690235 (+ 5034us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.690237 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.690332 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.690474 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.690671 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":70,"thread_start_us":116,"threads_started":1}
I20260811 02:47:14.702009 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.676186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60346 (local address 127.21.236.190:41631)
0811 02:47:14.676318 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.676323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.677175 (+ 852us) server_negotiation.cc:407] Connection header received
0811 02:47:14.677375 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.677379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.677446 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.677549 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.678153 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.679009 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.680535 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.680924 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.681012 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.696867 (+ 15855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.696876 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.696923 (+ 47us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.696977 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.699517 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.700285 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.700292 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.700297 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.700388 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.700709 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.700715 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.700719 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.701000 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.701075 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.701521 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.701664 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:47:14.702884 1636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.676341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.0.0.1:60346)
0811 02:47:14.677076 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.677092 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.677252 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.677569 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.677572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.677804 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.678000 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.678009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.679211 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.679216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.680381 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.680393 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.681172 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.681176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.681661 (+ 485us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.695872 (+ 14211us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.695910 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.696589 (+ 679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.699755 (+ 3166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.699762 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.699768 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.700148 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.700500 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.700506 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.700511 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.700599 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.701187 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.701192 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.701312 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.702445 (+ 1133us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.702626 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":106,"thread_start_us":116,"threads_started":1}
I20260811 02:47:14.711313 1603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.676974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37676 (local address 127.21.236.189:46833)
0811 02:47:14.677147 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.677154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.678120 (+ 966us) server_negotiation.cc:407] Connection header received
0811 02:47:14.678298 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.678301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.678354 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.678435 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.679147 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.680035 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.681245 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.681617 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.681698 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.705473 (+ 23775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.705477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.705479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.705508 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.707041 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.709726 (+ 2685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.709731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.709733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.709854 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.710202 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.710208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.710211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.710436 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.710511 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.710894 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.711090 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:47:14.711961 1635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.676743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:46833 (local address 127.0.0.1:37676)
0811 02:47:14.678034 (+ 1291us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.678048 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.678192 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.678591 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.678594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.678832 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.678965 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.678972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.680185 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.680188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.681103 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.681113 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.681852 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.681855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.681970 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.704798 (+ 22828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.704809 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.705273 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.709299 (+ 4026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.709302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.709304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.709547 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.709993 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.709996 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.709998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.710093 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.710600 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.710603 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.710697 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.711481 (+ 784us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.711686 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1219,"mutex_wait_us":2,"thread_start_us":126,"threads_started":1}
I20260811 02:47:14.718161 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:42169
--local_ip_for_outbound_sockets=127.21.236.129
--tserver_master_addrs=127.21.236.190:41631
--webserver_port=37229
--webserver_interface=127.21.236.129
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.129.keytab
--principal=oryx/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.236.190:41631,127.21.236.189:46833
--tserver_master_addrs=127.21.236.190:41631,127.21.236.189:46833,127.21.236.188:34191 with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:47:14.722519 1603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.716697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37688 (local address 127.21.236.189:46833)
0811 02:47:14.716818 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.716822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.717156 (+ 334us) server_negotiation.cc:407] Connection header received
0811 02:47:14.717340 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.717343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.717401 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.717476 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:14.717869 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.718615 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.720741 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.721507 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.721675 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.721847 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.722271 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.722370 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416433, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.190@KRBTEST.COM for oryx/127.21.236.188@KRBTEST.COM
I20260811 02:47:14.722995 1638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.716275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:46833 (local address 127.0.0.1:37688)
0811 02:47:14.717045 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.717061 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.717227 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.717639 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.717645 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.717668 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:14.717758 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.717764 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.718772 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.718775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.720594 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.720608 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.721963 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.721967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.722073 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.722145 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.722743 (+ 598us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.722842 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":3,"thread_start_us":109,"threads_started":1}
I20260811 02:47:14.723793 1415 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 02:47:14.724272 1461 sys_catalog.cc:455] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2070708acfb041e58e41726a399e81be. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } }
I20260811 02:47:14.724421 1461 sys_catalog.cc:458] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:47:14.727224 1324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.716234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:34191 (local address 127.0.0.1:45066)
0811 02:47:14.716603 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.716618 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.716711 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.717253 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.717259 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.717489 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.717609 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.717617 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.719049 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.719053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.719943 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.719957 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.720666 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.720671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.720772 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.721311 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.721329 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.723662 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.725635 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.725638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.725640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.725854 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.726160 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.726163 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.726165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.726256 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.726646 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.726649 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.726755 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.726870 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.727027 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":47}
I20260811 02:47:14.727223 1598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.716469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45066 (local address 127.21.236.188:34191)
0811 02:47:14.716600 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.716604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.716799 (+ 195us) server_negotiation.cc:407] Connection header received
0811 02:47:14.716909 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.716913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.716992 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.717109 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.717733 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.718539 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.720087 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.720431 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.720532 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.723831 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.723835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.723838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.723865 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.725494 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.725971 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.725975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.725979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.726065 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.726352 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.726357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.726358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.726545 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.726657 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.726920 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.727040 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 02:47:14.728502 417 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.236.188:34191: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf"
member_type: NON_VOTER
last_known_addr {
host: "127.21.236.188"
port: 34191
}
attrs {
promote: true
}
}
dest_uuid: "2070708acfb041e58e41726a399e81be"
cas_config_opid_index: 5
I20260811 02:47:14.729082 417 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } }
I20260811 02:47:14.730219 1330 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f60bba59016c4a489122f38ed2bf24bf (127.21.236.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } } }
I20260811 02:47:14.730942 1555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:14.732321 1330 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.236.188:34191
I20260811 02:47:14.732498 1555 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 FOLLOWER]: Refusing update from remote peer 2070708acfb041e58e41726a399e81be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 02:47:14.732502 1330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } } }
I20260811 02:47:14.732596 1330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:14.733140 1335 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [LEADER]: Connected to new peer: Peer: permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:14.734110 1415 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 LEARNER]: Refusing update from remote peer 2070708acfb041e58e41726a399e81be: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 02:47:14.734645 1335 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 02:47:14.736379 1415 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f60bba59016c4a489122f38ed2bf24bf (127.21.236.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } } }
I20260811 02:47:14.737433 1555 raft_consensus.cc:858] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } }
W20260811 02:47:14.737566 1555 raft_consensus.cc:1565] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } }
I20260811 02:47:14.738564 1352 master.cc:537] Master@127.21.236.188:34191 shutting down...
I20260811 02:47:14.739212 1461 sys_catalog.cc:455] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } } }
I20260811 02:47:14.739344 1461 sys_catalog.cc:458] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: This master's current role is: LEARNER
I20260811 02:47:14.740818 1335 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be: attempting to promote NON_VOTER e56f6424c0994b6f8ad2bbab85626f10 to VOTER
I20260811 02:47:14.740829 1555 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e56f6424c0994b6f8ad2bbab85626f10 (127.21.236.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } }
I20260811 02:47:14.741288 1335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } }
I20260811 02:47:14.743211 1606 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2070708acfb041e58e41726a399e81be. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } }
I20260811 02:47:14.743258 1415 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 LEARNER]: Refusing update from remote peer 2070708acfb041e58e41726a399e81be: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 02:47:14.743328 1606 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 02:47:14.743419 1606 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } }
I20260811 02:47:14.743479 1606 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 02:47:14.743538 1606 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: NON_VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: true } } }
I20260811 02:47:14.743597 1606 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 02:47:14.743868 1331 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260811 02:47:14.746196 395 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be -> Peer f60bba59016c4a489122f38ed2bf24bf (127.21.236.188:34191): Couldn't send request to peer f60bba59016c4a489122f38ed2bf24bf. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 02:47:14.747265 1644 mvcc.cc:205] Tried to move back new op lower bound from 7317161710667304960 to 7317161710378553344. Current Snapshot: MvccSnapshot[applied={T|T < 7317161710419161088}]
I20260811 02:47:14.748873 1352 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 02:47:14.749117 1352 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 02:47:14.749002 1336 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e56f6424c0994b6f8ad2bbab85626f10 (127.21.236.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } } }
I20260811 02:47:14.749197 1352 tablet_replica.cc:330] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: stopping tablet replica
I20260811 02:47:14.750491 1336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } } }
I20260811 02:47:14.750605 1336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:14.750769 1336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } } }
I20260811 02:47:14.750828 1336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:14.751464 1415 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e56f6424c0994b6f8ad2bbab85626f10 (127.21.236.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } } }
I20260811 02:47:14.753137 1461 sys_catalog.cc:455] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } } }
I20260811 02:47:14.753263 1461 sys_catalog.cc:458] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:47:14.760640 1352 master.cc:552] Master@127.21.236.188:34191 shutdown complete.
I20260811 02:47:14.760713 1352 master_runner.cc:305] Clearing existing system tablet
I20260811 02:47:14.760943 1352 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:14.761016 1352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:14.762285 1352 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 02:47:14.763202 1658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:14.763432 1352 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:14.763586 1352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 02:47:14.763624 1352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/wal
uuid: "f60bba59016c4a489122f38ed2bf24bf"
format_stamp: "Formatted at 2026-08-11 02:47:14 on dist-test-slave-n326"
server_key: "79a504765065fe739799c2e5844bebb4"
server_key_iv: "887dac1fe6145b7de5e049c89d8f0b29"
server_key_version: "encryptionkey@0"
I20260811 02:47:14.765152 1352 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:14.767768 1352 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 02:47:14.767863 1352 log.cc:1196] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 02:47:14.768157 1352 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: Deleting consensus metadata
I20260811 02:47:14.768316 1352 master_runner.cc:315] Copying system tablet from 127.21.236.190:41631
I20260811 02:47:14.769464 1352 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: tablet copy: Beginning tablet copy session from remote peer at address 127.21.236.190:41631
I20260811 02:47:14.779227 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.771563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.21.236.190:41631)
0811 02:47:14.771711 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.771716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.772029 (+ 313us) server_negotiation.cc:407] Connection header received
0811 02:47:14.772210 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.772213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.772260 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.772346 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.772808 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.773339 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.774373 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.774606 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.774738 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.776004 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.776007 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.776009 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.776035 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.777509 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.777964 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.777968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.777969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.778045 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.778342 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.778347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.778349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.778546 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.778657 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.778923 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.779034 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 02:47:14.779266 1671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.771477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.0.0.1:60358)
0811 02:47:14.771927 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.771942 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.772081 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.772375 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.772378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.772591 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.772684 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.772690 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.773451 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.773457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.774254 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.774264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.774751 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.774754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.774936 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.775508 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.775521 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.775842 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.777634 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.777639 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.777641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.777847 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.778137 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.778141 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.778143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.778232 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.778646 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.778649 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.778756 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.778867 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.779021 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":140,"threads_started":1}
I20260811 02:47:14.780100 448 tablet_copy_service.cc:140] P 2070708acfb041e58e41726a399e81be: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f60bba59016c4a489122f38ed2bf24bf ({username='oryx', principal='oryx/127.21.236.188@KRBTEST.COM'} at 127.0.0.1:60358)
I20260811 02:47:14.780215 448 tablet_copy_service.cc:161] P 2070708acfb041e58e41726a399e81be: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f60bba59016c4a489122f38ed2bf24bf at {username='oryx', principal='oryx/127.21.236.188@KRBTEST.COM'} at 127.0.0.1:60358: session id = f60bba59016c4a489122f38ed2bf24bf-00000000000000000000000000000000
I20260811 02:47:14.781050 448 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 02:47:14.782240 1352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 02:47:14.784273 1352 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: tablet copy: Starting download of 0 data blocks...
I20260811 02:47:14.784492 1352 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: tablet copy: Starting download of 1 WAL segments...
I20260811 02:47:14.786005 1352 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 02:47:14.787567 448 tablet_copy_service.cc:342] P 2070708acfb041e58e41726a399e81be: Request end of tablet copy session f60bba59016c4a489122f38ed2bf24bf-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.236.188@KRBTEST.COM'} at 127.0.0.1:60358
I20260811 02:47:14.787662 448 tablet_copy_service.cc:433] P 2070708acfb041e58e41726a399e81be: ending tablet copy session f60bba59016c4a489122f38ed2bf24bf-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f60bba59016c4a489122f38ed2bf24bf
I20260811 02:47:14.792044 944 heartbeater.cc:503] Master 127.21.236.190:41631 was elected leader, sending a full tablet report...
I20260811 02:47:14.794766 1352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:14.796669 1673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:14.797709 1352 server_base.cc:1034] running on GCE node
W20260811 02:47:14.797868 1677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:14.798378 1674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:14.798703 1352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:14.799000 1352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:14.800139 1352 hybrid_clock.cc:648] HybridClock initialized: now 1786416434800103 us; error 56 us; skew 500 ppm
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416434, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:14.801817 1352 init.cc:377] Logged in from keytab as oryx/127.21.236.188@KRBTEST.COM (short username oryx)
I20260811 02:47:14.802320 1352 webserver.cc:460] Webserver started at http://127.21.236.188:46659/ using document root <none> and password file <none>
I20260811 02:47:14.802534 1352 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:14.802591 1352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:14.803817 1352 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 02:47:14.804778 1685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:14.805035 1352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 02:47:14.805133 1352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/wal
uuid: "f60bba59016c4a489122f38ed2bf24bf"
format_stamp: "Formatted at 2026-08-11 02:47:14 on dist-test-slave-n326"
server_key: "79a504765065fe739799c2e5844bebb4"
server_key_iv: "887dac1fe6145b7de5e049c89d8f0b29"
server_key_version: "encryptionkey@0"
I20260811 02:47:14.805256 1352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:14.821866 1352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:14.824093 1352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:14.829183 1352 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.188:34191
I20260811 02:47:14.829190 1790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.188:34191 every 8 connection(s)
I20260811 02:47:14.829770 1352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/master-2/data/info.pb
I20260811 02:47:14.831311 1791 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:47:14.832170 1791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: Bootstrap starting.
I20260811 02:47:14.838948 1791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:47:14.839311 1791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: Bootstrap complete.
I20260811 02:47:14.839843 1791 raft_consensus.cc:348] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } }
I20260811 02:47:14.839983 1791 raft_consensus.cc:729] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f60bba59016c4a489122f38ed2bf24bf, State: Initialized, Role: LEARNER
I20260811 02:47:14.840104 1791 consensus_queue.cc:260] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } }
I20260811 02:47:14.840469 1794 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } } }
I20260811 02:47:14.840641 1794 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: This master's current role is: LEARNER
I20260811 02:47:14.840644 1791 sys_catalog.cc:564] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:14.844645 1809 catalog_manager.cc:1259] Loaded cluster ID: fe3205df127e4363a58f17badb54d6dd
I20260811 02:47:14.844726 1809 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: loading cluster ID for follower catalog manager: success
I20260811 02:47:14.846395 1809 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: acquiring CA information for follower catalog manager: success
I20260811 02:47:14.847625 1809 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 02:47:14.890288 1640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:14.890656 1640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:14.890723 1640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:14.896637 1640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:14.896709 1640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:14.896816 1640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:47:14.902801 1640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.129.keytab
--principal=oryx/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:42169
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=37229
--tserver_master_addrs=127.21.236.190:41631,127.21.236.189:46833,127.21.236.188:34191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:14.903987 1640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:14.905186 1640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:14.916157 1822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:14.916262 1640 server_base.cc:1034] running on GCE node
W20260811 02:47:14.916173 1819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:14.916432 1818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:14.916908 1640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:14.917585 1640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:14.918779 1640 hybrid_clock.cc:648] HybridClock initialized: now 1786416434918759 us; error 33 us; skew 500 ppm
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416434, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:14.921902 1640 init.cc:377] Logged in from keytab as oryx/127.21.236.129@KRBTEST.COM (short username oryx)
I20260811 02:47:14.922731 1640 webserver.cc:460] Webserver started at http://127.21.236.129:37229/ using document root <none> and password file <none>
I20260811 02:47:14.923444 1640 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:14.923545 1640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:14.927299 1640 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 02:47:14.929394 1837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:14.930708 1640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:47:14.930878 1640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "1e0426b9d6594109ba2a0afbdf86107d"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "a6aaadc7866c0bcd4738aca65580b6cd"
server_key_iv: "261a1339a7fde23308d7b5cdc10d795a"
server_key_version: "encryptionkey@0"
I20260811 02:47:14.931572 1640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:14.951598 1640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:14.954808 1640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:14.955011 1640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:14.955641 1640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:14.956492 1640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:14.956562 1640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:14.956632 1640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:14.956681 1640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:14.967010 1640 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:42169
I20260811 02:47:14.967032 2033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:42169 every 8 connection(s)
I20260811 02:47:14.968139 1640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:47:14.978037 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 1640
I20260811 02:47:14.978276 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 726
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416434, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416434, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.188@KRBTEST.COM
I20260811 02:47:14.990383 2043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.970205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:57645 (local address 127.21.236.188:34191)
0811 02:47:14.970462 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.970466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.971474 (+ 1008us) server_negotiation.cc:407] Connection header received
0811 02:47:14.972835 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.972839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.972909 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.972980 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.974440 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.975345 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.976992 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.977389 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.977514 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.982776 (+ 5262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.982783 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.982789 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.982827 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.987568 (+ 4741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.988310 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.988316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.988321 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.988423 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.988851 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.988856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.988858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.989116 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.989199 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.989936 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.990100 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":110,"threads_started":1}
I20260811 02:47:14.992110 2046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.970517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:34191 (local address 127.21.236.129:57645)
0811 02:47:14.971277 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.971346 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.972558 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.973171 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.973180 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.973731 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.974260 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.974268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.975461 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.975464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.976861 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.976873 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.977679 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.977683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.977831 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.980184 (+ 2353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.980205 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.982092 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.987735 (+ 5643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.987745 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.987768 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.988188 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.988538 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.988544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.988548 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.988747 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.989267 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.989277 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.989650 (+ 373us) client_negotiation.cc:769] Sending connection context
0811 02:47:14.990698 (+ 1048us) client_negotiation.cc:240] Negotiation successful
0811 02:47:14.991163 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":235,"spinlock_wait_cycles":16384,"thread_start_us":129,"threads_started":1}
I20260811 02:47:14.995993 2035 heartbeater.cc:347] Connected to a master server at 127.21.236.188:34191
I20260811 02:47:14.996254 2035 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:14.996856 2035 heartbeater.cc:511] Master 127.21.236.188:34191 requested a full tablet report, sending...
I20260811 02:47:14.998670 1727 ts_manager.cc:194] Registered new tserver with Master: 1e0426b9d6594109ba2a0afbdf86107d (127.21.236.129:42169)
I20260811 02:47:14.999969 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.972063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:50385 (local address 127.21.236.190:41631)
0811 02:47:14.972199 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.972202 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.972216 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:47:14.972814 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.972818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.972880 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.972961 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.974386 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.975262 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.976758 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.977094 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.977197 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.983270 (+ 6073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:14.983277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:14.983283 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:14.983321 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:14.997538 (+ 14217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.998167 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.998174 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.998179 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.998298 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:14.998652 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:14.998658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:14.998663 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:14.998951 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:14.999056 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:14.999652 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 02:47:14.999846 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 02:47:15.000515 2049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.971646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.129:50385)
0811 02:47:14.971931 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.971947 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.972580 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.973169 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.973176 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.973611 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.974231 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.974246 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.975698 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.975700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.976598 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.976607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.977389 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.977395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.977675 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:14.978494 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:14.978522 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:14.982596 (+ 4074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:14.997731 (+ 15135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.997734 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.997736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.998038 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.998429 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:14.998432 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:14.998434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:14.998533 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:14.999306 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:14.999309 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:14.999411 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 02:47:15.000139 (+ 728us) client_negotiation.cc:240] Negotiation successful
0811 02:47:15.000326 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":157,"spinlock_wait_cycles":1792,"thread_start_us":106,"threads_started":1}
I20260811 02:47:15.001202 2036 heartbeater.cc:347] Connected to a master server at 127.21.236.190:41631
I20260811 02:47:15.001297 2036 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:15.001539 2036 heartbeater.cc:511] Master 127.21.236.190:41631 requested a full tablet report, sending...
I20260811 02:47:15.002398 417 ts_manager.cc:194] Re-registered known tserver with Master: 1e0426b9d6594109ba2a0afbdf86107d (127.21.236.129:42169)
I20260811 02:47:15.003065 417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:50385
I20260811 02:47:15.003794 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:33221
--local_ip_for_outbound_sockets=127.21.236.130
--tserver_master_addrs=127.21.236.190:41631
--webserver_port=34761
--webserver_interface=127.21.236.130
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.130.keytab
--principal=oryx/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.236.190:41631,127.21.236.189:46833
--tserver_master_addrs=127.21.236.190:41631,127.21.236.189:46833,127.21.236.188:34191 with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:47:14 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416434, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.189@KRBTEST.COM
I20260811 02:47:15.009225 2045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.970517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:46833 (local address 127.21.236.129:60565)
0811 02:47:14.971440 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:14.971456 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:14.972558 (+ 1102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:14.973171 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:14.973180 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:14.973731 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:14.974240 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.974251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.975442 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.975445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.976861 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:14.976873 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.977388 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.977392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:14.977638 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.003599 (+ 25961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:15.003614 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:15.005211 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:15.007397 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:15.007400 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:15.007402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:15.007661 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:15.008036 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:15.008041 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:15.008046 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:15.008159 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:15.008679 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:15.008682 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:15.008761 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 02:47:15.008850 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 02:47:15.008998 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":339,"spinlock_wait_cycles":36352,"thread_start_us":162,"threads_started":1}
I20260811 02:47:15.009330 1603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:14.970025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:60565 (local address 127.21.236.189:46833)
0811 02:47:14.970165 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:14.970170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:14.971557 (+ 1387us) server_negotiation.cc:407] Connection header received
0811 02:47:14.972838 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:14.972842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:14.972909 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:14.972975 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:14.974444 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.975265 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.977019 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:14.977248 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:14.977366 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.005389 (+ 28023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:15.005394 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:15.005397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:15.005429 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:15.007273 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.007780 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.007787 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.007792 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.007893 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.008278 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.008284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.008289 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.008566 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:15.008681 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.008942 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.009103 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:47:15.010670 2038 heartbeater.cc:347] Connected to a master server at 127.21.236.189:46833
I20260811 02:47:15.010794 2038 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:15.011096 2038 heartbeater.cc:511] Master 127.21.236.189:46833 requested a full tablet report, sending...
I20260811 02:47:15.012501 1165 heartbeater.cc:503] Master 127.21.236.190:41631 was elected leader, sending a full tablet report...
I20260811 02:47:15.012491 1384 ts_manager.cc:194] Registered new tserver with Master: 1e0426b9d6594109ba2a0afbdf86107d (127.21.236.129:42169)
W20260811 02:47:15.122443 2051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:15.122752 2051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:15.122838 2051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 02:47:15.126922 1324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.123411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:34191 (local address 127.0.0.1:45076)
0811 02:47:15.123607 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:15.123625 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:15.123739 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:15.124043 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:15.124047 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:15.124070 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:15.124187 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.124193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.125000 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.125006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.125778 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.125788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.126513 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.126517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.126608 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.126633 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 02:47:15.126698 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 02:47:15.126782 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 02:47:15.126922 2043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.123514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45076 (local address 127.21.236.188:34191)
0811 02:47:15.123685 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.123691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.123707 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:47:15.123858 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.123865 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.123921 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.124026 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:15.124306 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.124876 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.125914 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.126373 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.126512 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.126680 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.126717 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.126789 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 02:47:15.127276 2051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:15.127343 2051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:15.127439 2051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:47:15.127674 1745 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 02:47:15.127966 1794 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2070708acfb041e58e41726a399e81be. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } } }
I20260811 02:47:15.128114 1794 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: This master's current role is: LEARNER
I20260811 02:47:15.133234 2051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.130.keytab
--principal=oryx/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:33221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=34761
--tserver_master_addrs=127.21.236.190:41631,127.21.236.189:46833,127.21.236.188:34191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:15.134276 2051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:15.135438 2051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:15.145800 2062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:15.145795 2057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:15.145800 2058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:15.146178 2051 server_base.cc:1034] running on GCE node
I20260811 02:47:15.146641 2051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:15.147367 2051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:15.148556 2051 hybrid_clock.cc:648] HybridClock initialized: now 1786416435148537 us; error 39 us; skew 500 ppm
Aug 11 02:47:15 dist-test-slave-n326 krb5kdc[344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416435, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:15.151474 2051 init.cc:377] Logged in from keytab as oryx/127.21.236.130@KRBTEST.COM (short username oryx)
I20260811 02:47:15.152567 2051 webserver.cc:460] Webserver started at http://127.21.236.130:34761/ using document root <none> and password file <none>
I20260811 02:47:15.153187 2051 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:15.153297 2051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:15.157274 2051 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:47:15.159713 2075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:15.161084 2051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:47:15.161238 2051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "0b645f57220a478cb25806d0c7da19e8"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "d9dc7bc9b6a812b924976d19c58181e0"
server_key_iv: "39b2b333cfdda2c28e8cf994128ba1f4"
server_key_version: "encryptionkey@0"
I20260811 02:47:15.162007 2051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:15.192101 2051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:15.195501 2051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:15.195735 2051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:15.196374 2051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:15.197314 2051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:15.197388 2051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:15.197463 2051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:15.197508 2051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:15.208058 2051 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:33221
I20260811 02:47:15.208092 2247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:33221 every 8 connection(s)
I20260811 02:47:15.209103 2051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:47:15.212666 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 2051
I20260811 02:47:15.212888 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 949
Aug 11 02:47:15 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416435, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.188@KRBTEST.COM
Aug 11 02:47:15 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416435, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.189@KRBTEST.COM
I20260811 02:47:15.225672 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:46517
--local_ip_for_outbound_sockets=127.21.236.131
--tserver_master_addrs=127.21.236.190:41631
--webserver_port=44991
--webserver_interface=127.21.236.131
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.131.keytab
--principal=oryx/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.236.190:41631,127.21.236.189:46833
--tserver_master_addrs=127.21.236.190:41631,127.21.236.189:46833,127.21.236.188:34191 with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:47:15 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416435, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
I20260811 02:47:15.229277 2043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.211175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:59553 (local address 127.21.236.188:34191)
0811 02:47:15.211327 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.211331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.212544 (+ 1213us) server_negotiation.cc:407] Connection header received
0811 02:47:15.214299 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.214303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.214359 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.214460 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:15.216135 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.217030 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.218860 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.219304 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.219402 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.224234 (+ 4832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:15.224238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:15.224241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:15.224269 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:15.226280 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.227029 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.227035 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.227038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.227169 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.227693 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.227702 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.227705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.227998 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:15.228105 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.228916 (+ 811us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.229069 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:47:15.229287 1603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.213053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:59407 (local address 127.21.236.189:46833)
0811 02:47:15.216188 (+ 3135us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.216192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.216206 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:47:15.216245 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.216247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.216285 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.216357 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:15.217088 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.217670 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.218888 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.219414 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.219562 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.224530 (+ 4968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:15.224537 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:15.224544 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:15.224582 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:15.226831 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.227409 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.227414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.227418 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.227531 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.227877 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.227882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.227886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.228135 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:15.228215 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.228901 (+ 686us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.229048 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3047}
I20260811 02:47:15.231225 2253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.211646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:34191 (local address 127.21.236.130:59553)
0811 02:47:15.212299 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:15.212383 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:15.213914 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:15.214671 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:15.214679 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:15.215246 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:15.215962 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.215980 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.217193 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.217200 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.218701 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.218717 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.219649 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.219655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.220057 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.221169 (+ 1112us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:15.221189 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:15.223988 (+ 2799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:15.226430 (+ 2442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:15.226441 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:15.226480 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:15.226881 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:15.227312 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:15.227315 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:15.227317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:15.227552 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:15.228193 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:15.228203 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:15.228588 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 02:47:15.229613 (+ 1025us) client_negotiation.cc:240] Negotiation successful
0811 02:47:15.230069 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":399,"spinlock_wait_cycles":16128,"thread_start_us":698,"threads_started":1}
I20260811 02:47:15.231555 2255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.212918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:46833 (local address 127.21.236.130:59407)
0811 02:47:15.213640 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:15.213653 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:15.214125 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:15.216425 (+ 2300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:15.216428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:15.216788 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:15.216920 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.216928 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.217802 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.217805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.218692 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.218702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.219782 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.219786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.220045 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.221042 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:15.221070 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:15.223991 (+ 2921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:15.226979 (+ 2988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:15.226985 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:15.226989 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:15.227280 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:15.227668 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:15.227673 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:15.227677 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:15.227773 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:15.228296 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:15.228300 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:15.228583 (+ 283us) client_negotiation.cc:769] Sending connection context
0811 02:47:15.229480 (+ 897us) client_negotiation.cc:240] Negotiation successful
0811 02:47:15.229969 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":103,"thread_start_us":102,"threads_started":1}
I20260811 02:47:15.231683 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.211889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:60507 (local address 127.21.236.190:41631)
0811 02:47:15.212067 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.212074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.219424 (+ 7350us) server_negotiation.cc:407] Connection header received
0811 02:47:15.219489 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.219493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.219557 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.219642 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:15.221116 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.222081 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.222982 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.223506 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.223622 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.226668 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:15.226672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:15.226675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:15.226708 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:15.228725 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.229272 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.229275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.229277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.229814 (+ 537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.230161 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.230165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.230166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.230401 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:15.230505 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.231355 (+ 850us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.231513 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 02:47:15.231238 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.211646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.130:60507)
0811 02:47:15.213392 (+ 1746us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:15.213409 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:15.213914 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:15.219783 (+ 5869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:15.219786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:15.220099 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:15.220438 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.220450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.222191 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.222198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.222855 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.222867 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.223870 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.223875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.223994 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.224538 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:15.224557 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:15.226493 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:15.228870 (+ 2377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:15.228873 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:15.228875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:15.229146 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:15.229921 (+ 775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:15.229924 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:15.229926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:15.230038 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:15.230564 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:15.230567 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:15.230662 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 02:47:15.230768 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 02:47:15.230915 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1650,"mutex_wait_us":197,"thread_start_us":192,"threads_started":1}
I20260811 02:47:15.233039 2251 heartbeater.cc:347] Connected to a master server at 127.21.236.189:46833
I20260811 02:47:15.233124 2250 heartbeater.cc:347] Connected to a master server at 127.21.236.190:41631
I20260811 02:47:15.233317 2251 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:15.233385 2248 heartbeater.cc:347] Connected to a master server at 127.21.236.188:34191
I20260811 02:47:15.233471 2248 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:15.233317 2250 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:15.233978 2251 heartbeater.cc:511] Master 127.21.236.189:46833 requested a full tablet report, sending...
I20260811 02:47:15.233970 2250 heartbeater.cc:511] Master 127.21.236.190:41631 requested a full tablet report, sending...
I20260811 02:47:15.233971 2248 heartbeater.cc:511] Master 127.21.236.188:34191 requested a full tablet report, sending...
I20260811 02:47:15.235371 1727 ts_manager.cc:194] Registered new tserver with Master: 0b645f57220a478cb25806d0c7da19e8 (127.21.236.130:33221)
I20260811 02:47:15.235450 417 ts_manager.cc:194] Re-registered known tserver with Master: 0b645f57220a478cb25806d0c7da19e8 (127.21.236.130:33221)
I20260811 02:47:15.235971 1384 ts_manager.cc:194] Registered new tserver with Master: 0b645f57220a478cb25806d0c7da19e8 (127.21.236.130:33221)
I20260811 02:47:15.236317 417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:60507
W20260811 02:47:15.342584 2259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:15.342883 2259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:15.342993 2259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:15.347195 2259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:15.347273 2259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:15.347366 2259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:47:15.352555 2259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.131.keytab
--principal=oryx/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:46517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=44991
--tserver_master_addrs=127.21.236.190:41631,127.21.236.189:46833,127.21.236.188:34191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:15.353642 2259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:15.354612 2259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:15.365031 2265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:15.365052 2270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:15.365037 2266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:15.365326 2259 server_base.cc:1034] running on GCE node
I20260811 02:47:15.365712 2259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:15.366385 2259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:15.367548 2259 hybrid_clock.cc:648] HybridClock initialized: now 1786416435367518 us; error 40 us; skew 500 ppm
Aug 11 02:47:15 dist-test-slave-n326 krb5kdc[344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416435, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:15.370364 2259 init.cc:377] Logged in from keytab as oryx/127.21.236.131@KRBTEST.COM (short username oryx)
I20260811 02:47:15.371223 2259 webserver.cc:460] Webserver started at http://127.21.236.131:44991/ using document root <none> and password file <none>
I20260811 02:47:15.371747 2259 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:15.371819 2259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:15.375554 2259 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 02:47:15.377550 2280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:15.378821 2259 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:47:15.378958 2259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "70311aa80d1841af979f44135d73f088"
format_stamp: "Formatted at 2026-08-11 02:47:13 on dist-test-slave-n326"
server_key: "79b557928b79e0c63eb21bf2585d741b"
server_key_iv: "f24ad2e3331b9db8deb94ba4e9b19286"
server_key_version: "encryptionkey@0"
I20260811 02:47:15.379544 2259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:15.398990 2259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:15.402629 2259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:15.402832 2259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:15.403512 2259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:15.404384 2259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:15.404456 2259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:15.404529 2259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:15.404587 2259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:15.413623 2259 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:46517
I20260811 02:47:15.413694 2485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:46517 every 8 connection(s)
I20260811 02:47:15.414644 2259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:47:15.424222 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 2259
Aug 11 02:47:15 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416435, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
Aug 11 02:47:15 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416435, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.189@KRBTEST.COM
Aug 11 02:47:15 dist-test-slave-n326 krb5kdc[344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416435, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.188@KRBTEST.COM
I20260811 02:47:15.431613 2043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.416452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:47463 (local address 127.21.236.188:34191)
0811 02:47:15.416592 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.416597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.417627 (+ 1030us) server_negotiation.cc:407] Connection header received
0811 02:47:15.418656 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.418659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.418726 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.418816 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:15.420310 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.420922 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.422148 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.422370 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.422499 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.426756 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:15.426759 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:15.426762 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:15.426788 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:15.429241 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.429760 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.429766 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.429770 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.429859 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.430200 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.430205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.430208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.430427 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:15.430541 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.431300 (+ 759us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.431436 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:47:15.432284 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.418009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:40339 (local address 127.21.236.190:41631)
0811 02:47:15.418136 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.418142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.418234 (+ 92us) server_negotiation.cc:407] Connection header received
0811 02:47:15.418944 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.418947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.418997 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.419111 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:15.420325 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.421147 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.422438 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.422756 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.422855 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.427403 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:15.427407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:15.427409 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:15.427445 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:15.429399 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.430939 (+ 1540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.430945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.430950 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.431054 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.431407 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.431411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.431413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.431612 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:15.431689 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.432041 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.432153 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:47:15.432477 2499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.416870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:34191 (local address 127.21.236.131:47463)
0811 02:47:15.417450 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:15.417513 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:15.418432 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:15.418980 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:15.418987 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:15.419606 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:15.420162 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.420176 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.421071 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.421074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.422010 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.422032 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.422534 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.422538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.422735 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.423458 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:15.423484 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:15.426594 (+ 3110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:15.429378 (+ 2784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:15.429381 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:15.429384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:15.429643 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:15.429974 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:15.429980 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:15.429983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:15.430081 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:15.430580 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:15.430584 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:15.430800 (+ 216us) client_negotiation.cc:769] Sending connection context
0811 02:47:15.431001 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 02:47:15.431862 (+ 861us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":642,"spinlock_wait_cycles":50432,"thread_start_us":704,"threads_started":1}
I20260811 02:47:15.432771 2502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.417905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.131:40339)
0811 02:47:15.418136 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:15.418151 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:15.418449 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:15.419146 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:15.419150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:15.419713 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:15.420176 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.420185 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.421295 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.421297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.422309 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.422320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.422871 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.422875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.422980 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.423606 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:15.423626 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:15.427234 (+ 3608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:15.430573 (+ 3339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:15.430577 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:15.430579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:15.430815 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:15.431201 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:15.431205 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:15.431207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:15.431295 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:15.431766 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:15.431768 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:15.431849 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 02:47:15.432472 (+ 623us) client_negotiation.cc:240] Negotiation successful
0811 02:47:15.432616 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":144,"spinlock_wait_cycles":20096,"thread_start_us":76,"threads_started":1}
I20260811 02:47:15.432850 1603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.416452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:43183 (local address 127.21.236.189:46833)
0811 02:47:15.416588 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.416593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.417960 (+ 1367us) server_negotiation.cc:407] Connection header received
0811 02:47:15.418653 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.418657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.418712 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.418769 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:15.420325 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.421147 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.422739 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.423071 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.423200 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.426328 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:15.426332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:15.426335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:15.426365 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:15.428826 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.429477 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.429482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.429484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.429576 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.430017 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.430023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.430027 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.430284 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:15.430376 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.432575 (+ 2199us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.432691 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 02:47:15.432474 2500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.416872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:46833 (local address 127.21.236.131:43183)
0811 02:47:15.417781 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:15.417796 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:15.418441 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:15.418978 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:15.418985 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:15.419610 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:15.420179 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.420185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.421303 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.421306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.422614 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.422623 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.423226 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.423230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.423324 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.423761 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:15.423777 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:15.426104 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:15.428956 (+ 2852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:15.428964 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:15.428980 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:15.429362 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:15.429693 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:15.429697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:15.429700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:15.429897 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:15.430431 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:15.430441 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:15.430800 (+ 359us) client_negotiation.cc:769] Sending connection context
0811 02:47:15.431000 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 02:47:15.431380 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"mutex_wait_us":205,"spinlock_wait_cycles":72064,"thread_start_us":168,"threads_started":1}
I20260811 02:47:15.433848 2486 heartbeater.cc:347] Connected to a master server at 127.21.236.188:34191
I20260811 02:47:15.433904 2491 heartbeater.cc:347] Connected to a master server at 127.21.236.189:46833
I20260811 02:47:15.434093 2486 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:15.434091 2491 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:15.434118 2489 heartbeater.cc:347] Connected to a master server at 127.21.236.190:41631
I20260811 02:47:15.434252 2489 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:15.434731 2491 heartbeater.cc:511] Master 127.21.236.189:46833 requested a full tablet report, sending...
I20260811 02:47:15.434731 2489 heartbeater.cc:511] Master 127.21.236.190:41631 requested a full tablet report, sending...
I20260811 02:47:15.434731 2486 heartbeater.cc:511] Master 127.21.236.188:34191 requested a full tablet report, sending...
I20260811 02:47:15.436133 417 ts_manager.cc:194] Re-registered known tserver with Master: 70311aa80d1841af979f44135d73f088 (127.21.236.131:46517)
I20260811 02:47:15.436156 1384 ts_manager.cc:194] Registered new tserver with Master: 70311aa80d1841af979f44135d73f088 (127.21.236.131:46517)
I20260811 02:47:15.436143 1727 ts_manager.cc:194] Registered new tserver with Master: 70311aa80d1841af979f44135d73f088 (127.21.236.131:46517)
I20260811 02:47:15.436905 417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:40339
I20260811 02:47:15.442240 2509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.426529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45078 (local address 127.21.236.188:34191)
0811 02:47:15.427221 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.427226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.428896 (+ 1670us) server_negotiation.cc:407] Connection header received
0811 02:47:15.429090 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.429093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.429142 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.429204 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:15.432044 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.432750 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.435114 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.435459 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.435559 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.438001 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:15.438005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:15.438007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:15.438033 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:15.439979 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.440576 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.440582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.440584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.440674 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.441017 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.441021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.441023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.441269 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:15.441432 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.441876 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.442020 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":103,"threads_started":1}
I20260811 02:47:15.444415 2514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.426914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60372 (local address 127.21.236.190:41631)
0811 02:47:15.428351 (+ 1437us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.428358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.428375 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:47:15.428414 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.428417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.428466 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.428559 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:15.430539 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.431371 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.438509 (+ 7138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.438868 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.438941 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.440550 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:15.440556 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:15.440560 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:15.440606 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:15.442408 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.443108 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.443114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.443116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.443214 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.443528 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.443533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.443534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.443759 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:15.443903 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.444141 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.444263 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1348,"thread_start_us":69,"threads_started":1}
I20260811 02:47:15.446259 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.427124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37698 (local address 127.21.236.189:46833)
0811 02:47:15.427376 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.427380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.430282 (+ 2902us) server_negotiation.cc:407] Connection header received
0811 02:47:15.435217 (+ 4935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.435220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.435266 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.435323 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:15.438465 (+ 3142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.439252 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.440537 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.440829 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.440957 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.442615 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:15.442619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:15.442623 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:15.442673 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:15.444295 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.444740 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.444743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.444745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.444833 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.445085 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.445089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.445090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.445291 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:15.445417 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.445653 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.445778 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":74,"threads_started":1}
I20260811 02:47:15.447587 417 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60372:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:47:15.450585 417 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:47:15.453235 2521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: NON_VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: true } } }
I20260811 02:47:15.453390 2521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:15.455226 2522 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be: attempting to promote NON_VOTER f60bba59016c4a489122f38ed2bf24bf to VOTER
I20260811 02:47:15.455524 2522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } }
I20260811 02:47:15.456768 1745 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 LEARNER]: Refusing update from remote peer 2070708acfb041e58e41726a399e81be: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 02:47:15.456768 1415 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 FOLLOWER]: Refusing update from remote peer 2070708acfb041e58e41726a399e81be: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 02:47:15.457135 2530 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 02:47:15.457268 2521 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [LEADER]: Connected to new peer: Peer: permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 02:47:15.458833 1335 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f60bba59016c4a489122f38ed2bf24bf (127.21.236.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } } }
I20260811 02:47:15.459335 1745 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f60bba59016c4a489122f38ed2bf24bf (127.21.236.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } } }
I20260811 02:47:15.460309 1335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } } }
I20260811 02:47:15.460436 1335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:15.460570 1335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } } }
I20260811 02:47:15.460649 1335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:15.460743 1335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } } }
I20260811 02:47:15.460813 1335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:15.461052 1794 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } } }
I20260811 02:47:15.461192 1794 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:47:15.461481 1415 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f60bba59016c4a489122f38ed2bf24bf (127.21.236.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } } }
I20260811 02:47:15.463857 2536 sys_catalog.cc:455] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2070708acfb041e58e41726a399e81be" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } } }
I20260811 02:47:15.463985 2536 sys_catalog.cc:458] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:47:15.472024 1324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.466318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42169 (local address 127.0.0.1:38992)
0811 02:47:15.466425 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:15.466440 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:15.466519 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:15.467686 (+ 1167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:15.467689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:15.467707 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:15.467810 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.467819 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.469224 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.469227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.470447 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.470454 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.471574 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.471578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.471664 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.471690 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 02:47:15.471776 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 02:47:15.471855 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":105}
I20260811 02:47:15.472577 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.466780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:33221 (local address 127.0.0.1:41264)
0811 02:47:15.467084 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:15.467102 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:15.467194 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:15.467951 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:15.467954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:15.467972 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:15.468067 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.468076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.469136 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.469142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.470428 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.470440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.472136 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.472141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.472242 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.472268 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 02:47:15.472342 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 02:47:15.472410 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":50,"thread_start_us":113,"threads_started":1}
I20260811 02:47:15.472625 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.467231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41264 (local address 127.21.236.130:33221)
0811 02:47:15.467584 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.467595 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.467623 (+ 28us) server_negotiation.cc:407] Connection header received
0811 02:47:15.467684 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.467688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.467824 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.467973 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:15.468164 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.469030 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.471193 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.471843 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.471954 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.472071 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.472391 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.472484 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":121,"threads_started":1}
I20260811 02:47:15.473163 2542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.466945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48674 (local address 127.21.236.131:46517)
0811 02:47:15.468022 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.468035 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.468057 (+ 22us) server_negotiation.cc:407] Connection header received
0811 02:47:15.468137 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.468142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.468314 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.468459 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:15.468691 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.469702 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.471313 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.472059 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.472206 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.472316 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.472897 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.473012 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":970,"thread_start_us":149,"threads_started":1}
I20260811 02:47:15.473451 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.466780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:46517 (local address 127.0.0.1:48674)
0811 02:47:15.467366 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:15.467397 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:15.467477 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:15.468456 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:15.468459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:15.468479 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:15.468575 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.468581 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.470052 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.470056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.471173 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.471184 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.472380 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.472383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.472472 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.472508 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 02:47:15.472579 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 02:47:15.473333 (+ 754us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":25,"thread_start_us":101,"threads_started":1}
I20260811 02:47:15.472049 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.466508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38992 (local address 127.21.236.129:42169)
0811 02:47:15.467163 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.467180 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.467205 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:47:15.467287 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.467293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.467532 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.467723 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:15.467967 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.469134 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.470567 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.471453 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.471586 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.471724 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.471805 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.471899 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":80,"threads_started":1}
I20260811 02:47:15.475982 1917 tablet_service.cc:1467] Processing CreateTablet for tablet 0311e24fbfef4aca81a811223ec2fce1 (DEFAULT_TABLE table=test-table [id=49c85a391e0b42c3ad6a7cecb19700eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:15.476025 2147 tablet_service.cc:1467] Processing CreateTablet for tablet 0311e24fbfef4aca81a811223ec2fce1 (DEFAULT_TABLE table=test-table [id=49c85a391e0b42c3ad6a7cecb19700eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:15.476015 2366 tablet_service.cc:1467] Processing CreateTablet for tablet 0311e24fbfef4aca81a811223ec2fce1 (DEFAULT_TABLE table=test-table [id=49c85a391e0b42c3ad6a7cecb19700eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:15.476926 1917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0311e24fbfef4aca81a811223ec2fce1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:15.477010 2366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0311e24fbfef4aca81a811223ec2fce1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:15.477010 2147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0311e24fbfef4aca81a811223ec2fce1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:15.484372 2546 tablet_bootstrap.cc:492] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d: Bootstrap starting.
I20260811 02:47:15.485446 2547 tablet_bootstrap.cc:492] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8: Bootstrap starting.
I20260811 02:47:15.487217 2546 tablet_bootstrap.cc:654] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:15.487403 2548 tablet_bootstrap.cc:492] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088: Bootstrap starting.
I20260811 02:47:15.487660 2547 tablet_bootstrap.cc:654] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:15.488174 2546 log.cc:824] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:15.488440 2547 log.cc:824] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:15.489413 2548 tablet_bootstrap.cc:654] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:15.490211 2546 tablet_bootstrap.cc:492] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d: No bootstrap required, opened a new log
I20260811 02:47:15.490212 2548 log.cc:824] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:15.490420 2546 ts_tablet_manager.cc:1397] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260811 02:47:15.490674 2547 tablet_bootstrap.cc:492] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8: No bootstrap required, opened a new log
I20260811 02:47:15.490873 2547 ts_tablet_manager.cc:1397] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:47:15.492164 2548 tablet_bootstrap.cc:492] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088: No bootstrap required, opened a new log
I20260811 02:47:15.492341 2548 ts_tablet_manager.cc:1397] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:47:15.494558 2546 raft_consensus.cc:348] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } } peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } }
I20260811 02:47:15.494843 2546 raft_consensus.cc:374] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:15.494691 2547 raft_consensus.cc:348] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } } peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } }
I20260811 02:47:15.494901 2546 raft_consensus.cc:729] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e0426b9d6594109ba2a0afbdf86107d, State: Initialized, Role: FOLLOWER
I20260811 02:47:15.494915 2547 raft_consensus.cc:374] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:15.494974 2547 raft_consensus.cc:729] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b645f57220a478cb25806d0c7da19e8, State: Initialized, Role: FOLLOWER
I20260811 02:47:15.495409 2547 consensus_queue.cc:260] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } } peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } }
I20260811 02:47:15.495434 2546 consensus_queue.cc:260] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } } peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } }
I20260811 02:47:15.496106 2250 heartbeater.cc:503] Master 127.21.236.190:41631 was elected leader, sending a full tablet report...
I20260811 02:47:15.496421 2547 ts_tablet_manager.cc:1428] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260811 02:47:15.496860 2546 ts_tablet_manager.cc:1428] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:47:15.498261 2548 raft_consensus.cc:348] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } } peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } }
I20260811 02:47:15.498505 2548 raft_consensus.cc:374] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:15.498569 2548 raft_consensus.cc:729] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70311aa80d1841af979f44135d73f088, State: Initialized, Role: FOLLOWER
I20260811 02:47:15.498986 2548 consensus_queue.cc:260] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } } peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } }
I20260811 02:47:15.500339 2548 ts_tablet_manager.cc:1428] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088: Time spent starting tablet: real 0.008s user 0.000s sys 0.007s
I20260811 02:47:15.500694 2489 heartbeater.cc:503] Master 127.21.236.190:41631 was elected leader, sending a full tablet report...
I20260811 02:47:15.500931 2036 heartbeater.cc:503] Master 127.21.236.190:41631 was elected leader, sending a full tablet report...
I20260811 02:47:15.519825 2561 raft_consensus.cc:482] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:47:15.520021 2561 raft_consensus.cc:504] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } } peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } }
I20260811 02:47:15.521299 2561 leader_election.cc:288] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e0426b9d6594109ba2a0afbdf86107d (127.21.236.129:42169), 0b645f57220a478cb25806d0c7da19e8 (127.21.236.130:33221)
I20260811 02:47:15.525099 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.521845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:35751 (local address 127.21.236.130:33221)
0811 02:47:15.521977 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.521980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.521991 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:47:15.522035 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.522037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.522073 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.522161 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:15.522420 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.522905 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.524087 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.524512 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.524649 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.524728 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.524896 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.524967 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:47:15.525111 2502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.521567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:33221 (local address 127.21.236.131:35751)
0811 02:47:15.521809 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:15.521822 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:15.521926 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:15.522170 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:15.522173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:15.522189 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:15.522299 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.522305 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.523011 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.523025 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.523957 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.523966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.524638 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.524641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.524737 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.524768 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 02:47:15.524868 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 02:47:15.524952 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":45}
I20260811 02:47:15.525118 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.521633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:49255 (local address 127.21.236.129:42169)
0811 02:47:15.521761 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.521766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.521781 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:47:15.521919 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.521922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.521977 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.522081 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:15.522329 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.522823 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.524098 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.524530 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.524670 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.524722 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.524896 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.524962 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:47:15.525118 2500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.521477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42169 (local address 127.21.236.131:49255)
0811 02:47:15.521697 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:15.521713 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:15.521800 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:15.522082 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:15.522085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:15.522100 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:15.522224 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.522229 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.522927 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.522932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.523956 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:15.523967 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.524659 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.524663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:15.524768 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.524788 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 02:47:15.524883 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 02:47:15.524954 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":54}
I20260811 02:47:15.525760 1963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0311e24fbfef4aca81a811223ec2fce1" candidate_uuid: "70311aa80d1841af979f44135d73f088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e0426b9d6594109ba2a0afbdf86107d" is_pre_election: true
I20260811 02:47:15.525784 2176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0311e24fbfef4aca81a811223ec2fce1" candidate_uuid: "70311aa80d1841af979f44135d73f088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b645f57220a478cb25806d0c7da19e8" is_pre_election: true
I20260811 02:47:15.526062 2176 raft_consensus.cc:2454] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70311aa80d1841af979f44135d73f088 in term 0.
I20260811 02:47:15.526083 1963 raft_consensus.cc:2454] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70311aa80d1841af979f44135d73f088 in term 0.
I20260811 02:47:15.526592 2287 leader_election.cc:302] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b645f57220a478cb25806d0c7da19e8, 1e0426b9d6594109ba2a0afbdf86107d, 70311aa80d1841af979f44135d73f088; no voters:
I20260811 02:47:15.526896 2561 raft_consensus.cc:2781] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:47:15.526984 2561 raft_consensus.cc:482] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:47:15.527029 2561 raft_consensus.cc:3037] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:15.528138 2561 raft_consensus.cc:504] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } } peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } }
I20260811 02:47:15.528455 2561 leader_election.cc:288] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [CANDIDATE]: Term 1 election: Requested vote from peers 1e0426b9d6594109ba2a0afbdf86107d (127.21.236.129:42169), 0b645f57220a478cb25806d0c7da19e8 (127.21.236.130:33221)
I20260811 02:47:15.528805 2176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0311e24fbfef4aca81a811223ec2fce1" candidate_uuid: "70311aa80d1841af979f44135d73f088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b645f57220a478cb25806d0c7da19e8"
I20260811 02:47:15.528805 1963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0311e24fbfef4aca81a811223ec2fce1" candidate_uuid: "70311aa80d1841af979f44135d73f088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e0426b9d6594109ba2a0afbdf86107d"
I20260811 02:47:15.528939 2176 raft_consensus.cc:3037] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:15.528939 1963 raft_consensus.cc:3037] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:15.530229 2176 raft_consensus.cc:2454] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70311aa80d1841af979f44135d73f088 in term 1.
I20260811 02:47:15.530229 1963 raft_consensus.cc:2454] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70311aa80d1841af979f44135d73f088 in term 1.
I20260811 02:47:15.530558 2287 leader_election.cc:302] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b645f57220a478cb25806d0c7da19e8, 70311aa80d1841af979f44135d73f088; no voters:
I20260811 02:47:15.530784 2561 raft_consensus.cc:2781] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:15.531052 2561 raft_consensus.cc:686] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [term 1 LEADER]: Becoming Leader. State: Replica: 70311aa80d1841af979f44135d73f088, State: Running, Role: LEADER
I20260811 02:47:15.531539 2561 consensus_queue.cc:237] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } } peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } }
I20260811 02:47:15.535182 410 catalog_manager.cc:5495] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70311aa80d1841af979f44135d73f088 (127.21.236.131). New cstate: current_term: 1 leader_uuid: "70311aa80d1841af979f44135d73f088" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } health_report { overall_health: HEALTHY } } }
I20260811 02:47:15.558714 2542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.555556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48682 (local address 127.21.236.131:46517)
0811 02:47:15.555671 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.555674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.555688 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:47:15.555795 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.555797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.555850 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.555937 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:47:15.556119 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.556607 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.557393 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.557622 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.557700 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.557947 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:47:15.558350 (+ 403us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:47:15.558437 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.558535 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.558584 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 02:47:15.564806 1963 raft_consensus.cc:1261] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d [term 1 FOLLOWER]: Refusing update from remote peer 70311aa80d1841af979f44135d73f088: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:15.564806 2176 raft_consensus.cc:1261] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8 [term 1 FOLLOWER]: Refusing update from remote peer 70311aa80d1841af979f44135d73f088: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:15.565563 2561 consensus_queue.cc:1035] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:15.565757 2562 consensus_queue.cc:1035] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:15.577500 2568 mvcc.cc:205] Tried to move back new op lower bound from 7317161720066543616 to 7317161719940055040. Current Snapshot: MvccSnapshot[applied={T|T < 7317161720066543616 or (T in {7317161720066543616})}]
I20260811 02:47:15.586408 411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:15.586566 411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60372:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:15.591361 411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2070708acfb041e58e41726a399e81be: Sending DeleteTablet for 3 replicas of tablet 0311e24fbfef4aca81a811223ec2fce1
I20260811 02:47:15.592546 1917 tablet_service.cc:1514] Processing DeleteTablet for tablet 0311e24fbfef4aca81a811223ec2fce1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:15 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:38992
I20260811 02:47:15.592581 2366 tablet_service.cc:1514] Processing DeleteTablet for tablet 0311e24fbfef4aca81a811223ec2fce1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:15 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:48674
I20260811 02:47:15.593366 2147 tablet_service.cc:1514] Processing DeleteTablet for tablet 0311e24fbfef4aca81a811223ec2fce1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:15 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:41264
I20260811 02:47:15.593748 2583 tablet_replica.cc:330] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088: stopping tablet replica
I20260811 02:47:15.593853 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 366
I20260811 02:47:15.605726 2583 raft_consensus.cc:2229] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:47:15.606194 2583 raft_consensus.cc:2258] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:15.607978 2583 ts_tablet_manager.cc:1905] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:15.611501 2587 tablet_replica.cc:330] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8: stopping tablet replica
I20260811 02:47:15.611778 2583 ts_tablet_manager.cc:1918] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:15.611878 2583 log.cc:1196] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/0311e24fbfef4aca81a811223ec2fce1
I20260811 02:47:15.611891 2587 raft_consensus.cc:2229] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:15.612243 2583 ts_tablet_manager.cc:1939] T 0311e24fbfef4aca81a811223ec2fce1 P 70311aa80d1841af979f44135d73f088: Deleting consensus metadata
I20260811 02:47:15.613353 2584 tablet_replica.cc:330] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d: stopping tablet replica
I20260811 02:47:15.613716 2584 raft_consensus.cc:2229] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:15.614032 2584 raft_consensus.cc:2258] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 02:47:15.614199 2283 connection.cc:535] server connection from 127.0.0.1:48674 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 02:47:15.615584 2587 raft_consensus.cc:2258] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:15.621551 2584 ts_tablet_manager.cc:1905] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:15.622522 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.593965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37706 (local address 127.21.236.189:46833)
0811 02:47:15.601492 (+ 7527us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.601499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.601519 (+ 20us) server_negotiation.cc:407] Connection header received
0811 02:47:15.601570 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.601576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.601637 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.601733 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:15.611250 (+ 9517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.612125 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.613162 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.613544 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.613670 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.615728 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:15.615733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:15.615737 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:15.615767 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:15.618254 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.618817 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.618821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.618823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.618915 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.619340 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.619344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.619346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.619612 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:15.622132 (+ 2520us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.622218 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.622352 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7383}
I20260811 02:47:15.625644 2587 ts_tablet_manager.cc:1905] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:15.628396 2043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.618086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45090 (local address 127.21.236.188:34191)
0811 02:47:15.618209 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.618214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.618462 (+ 248us) server_negotiation.cc:407] Connection header received
0811 02:47:15.618669 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.618672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.618726 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.618783 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:15.619674 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.620459 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.621648 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.622002 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.622080 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.624414 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:15.624421 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:15.624427 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:15.624462 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:15.626270 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.626801 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.626805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.626808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.626900 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.627297 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.627301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.627302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.627541 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:15.627661 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.628077 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.628208 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":51,"spinlock_wait_cycles":1920}
I20260811 02:47:15.629055 2587 ts_tablet_manager.cc:1918] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:15.629173 2587 log.cc:1196] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/0311e24fbfef4aca81a811223ec2fce1
I20260811 02:47:15.629580 2587 ts_tablet_manager.cc:1939] T 0311e24fbfef4aca81a811223ec2fce1 P 0b645f57220a478cb25806d0c7da19e8: Deleting consensus metadata
I20260811 02:47:15.629693 1603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.618722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37722 (local address 127.21.236.189:46833)
0811 02:47:15.618832 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.618836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.619726 (+ 890us) server_negotiation.cc:407] Connection header received
0811 02:47:15.619929 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.619932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.619976 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.620025 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:15.621320 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.622077 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.623539 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.623935 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.624022 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.625959 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:15.625963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:15.625966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:15.625994 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:15.627732 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.628194 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.628197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.628199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.628278 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.628605 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.628608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.628610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.628823 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:15.628931 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.629396 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.629508 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 02:47:15.631027 2078 connection.cc:441] server connection from 127.0.0.1:41264 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41264 (request call id 1) could send its response
I20260811 02:47:15.631671 2584 ts_tablet_manager.cc:1918] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:15.631811 2584 log.cc:1196] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-0/wal/wals/0311e24fbfef4aca81a811223ec2fce1
I20260811 02:47:15.632131 2509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:15.593959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45080 (local address 127.21.236.188:34191)
0811 02:47:15.601485 (+ 7526us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:15.601492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:15.620474 (+ 18982us) server_negotiation.cc:407] Connection header received
0811 02:47:15.620543 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:15.620549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:15.620599 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:15.620677 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:15.621639 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.622631 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.624019 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:15.624389 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:15.624519 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:15.627329 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:15.627336 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:15.627343 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:15.627375 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:15.629539 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.630486 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.630490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.630492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.630585 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:15.631009 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:15.631014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:15.631029 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:15.631407 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:15.631509 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:15.631819 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 02:47:15.631939 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":215,"server-negotiator.queue_time_us":7389}
I20260811 02:47:15.632217 2584 ts_tablet_manager.cc:1939] T 0311e24fbfef4aca81a811223ec2fce1 P 1e0426b9d6594109ba2a0afbdf86107d: Deleting consensus metadata
W20260811 02:47:15.633431 1842 connection.cc:441] server connection from 127.0.0.1:38992 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38992 (request call id 1) could send its response
I20260811 02:47:16.580911 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:16.580313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.130:53199)
0811 02:47:16.580653 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:16.580775 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":159,"threads_started":1}
W20260811 02:47:16.581303 2250 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:41631 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
I20260811 02:47:16.581648 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:16.581182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.129:41159)
0811 02:47:16.581454 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:16.581550 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":72,"threads_started":1}
W20260811 02:47:16.582075 2036 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:41631 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
I20260811 02:47:16.582827 2602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:16.582370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.131:51429)
0811 02:47:16.582663 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:16.582750 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":108,"threads_started":1}
W20260811 02:47:16.583184 2489 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:41631 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
I20260811 02:47:17.091436 2603 raft_consensus.cc:482] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2070708acfb041e58e41726a399e81be)
I20260811 02:47:17.091702 2603 raft_consensus.cc:504] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } }
I20260811 02:47:17.092185 2606 raft_consensus.cc:482] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2070708acfb041e58e41726a399e81be)
I20260811 02:47:17.092368 2606 raft_consensus.cc:504] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } }
I20260811 02:47:17.092950 2603 leader_election.cc:288] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2070708acfb041e58e41726a399e81be (127.21.236.190:41631), e56f6424c0994b6f8ad2bbab85626f10 (127.21.236.189:46833)
I20260811 02:47:17.093356 2606 leader_election.cc:288] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2070708acfb041e58e41726a399e81be (127.21.236.190:41631), f60bba59016c4a489122f38ed2bf24bf (127.21.236.188:34191)
I20260811 02:47:17.093881 2611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.093187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.0.0.1:32930)
0811 02:47:17.093671 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.093765 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":64,"threads_started":1}
W20260811 02:47:17.094094 1690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
I20260811 02:47:17.094271 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.093514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.0.0.1:32934)
0811 02:47:17.093892 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.094099 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":74,"threads_started":1}
W20260811 02:47:17.094563 1380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
I20260811 02:47:17.095050 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.094771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.0.0.1:32950)
0811 02:47:17.094913 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.094962 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 02:47:17.095165 1690 leader_election.cc:334] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2070708acfb041e58e41726a399e81be (127.21.236.190:41631): Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
I20260811 02:47:17.095796 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.095549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.0.0.1:32960)
0811 02:47:17.095677 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.095731 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 02:47:17.096004 1380 leader_election.cc:334] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2070708acfb041e58e41726a399e81be (127.21.236.190:41631): Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
I20260811 02:47:17.098315 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.093877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60066 (local address 127.21.236.188:34191)
0811 02:47:17.094387 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:17.094392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:17.094404 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:47:17.094449 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:17.094453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:17.094510 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:17.094904 (+ 394us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:17.094998 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.095766 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.097369 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.097761 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.097885 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.097939 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:17.098124 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 02:47:17.098199 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":62,"threads_started":1}
I20260811 02:47:17.098315 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.093538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:34191 (local address 127.0.0.1:60066)
0811 02:47:17.093942 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.093959 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:17.094077 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:17.094649 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:17.094653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:17.094675 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:17.094792 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:17.094800 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.095878 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.095884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:17.097214 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:17.097225 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.097881 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.097885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:17.097976 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.098013 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 02:47:17.098107 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 02:47:17.098187 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":60,"threads_started":1}
I20260811 02:47:17.098500 2611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.093544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:46833 (local address 127.0.0.1:37574)
0811 02:47:17.094026 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.094039 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:17.094140 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:17.094590 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:17.094595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:17.094621 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:17.094753 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:17.094764 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.095955 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.095958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:17.097214 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:17.097225 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.098007 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.098011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:17.098118 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.098150 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 02:47:17.098275 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 02:47:17.098357 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":78,"threads_started":1}
I20260811 02:47:17.098850 1745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e56f6424c0994b6f8ad2bbab85626f10" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f60bba59016c4a489122f38ed2bf24bf" is_pre_election: true
I20260811 02:47:17.099093 1745 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56f6424c0994b6f8ad2bbab85626f10 in term 1.
I20260811 02:47:17.099478 1376 leader_election.cc:302] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56f6424c0994b6f8ad2bbab85626f10, f60bba59016c4a489122f38ed2bf24bf; no voters: 2070708acfb041e58e41726a399e81be
I20260811 02:47:17.099495 2620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.093735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37574 (local address 127.21.236.189:46833)
0811 02:47:17.094286 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:17.094293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:17.094310 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:47:17.094362 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:17.094365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:17.094424 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:17.094885 (+ 461us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:17.095012 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.095842 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.097350 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.097841 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.098020 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.098073 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:17.098586 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 02:47:17.098668 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":50,"threads_started":1}
I20260811 02:47:17.099941 2606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 02:47:17.100016 2606 raft_consensus.cc:482] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2070708acfb041e58e41726a399e81be)
I20260811 02:47:17.100059 2606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:47:17.099988 1415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f60bba59016c4a489122f38ed2bf24bf" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e56f6424c0994b6f8ad2bbab85626f10" is_pre_election: true
I20260811 02:47:17.101122 2606 raft_consensus.cc:504] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } }
I20260811 02:47:17.101373 1415 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f60bba59016c4a489122f38ed2bf24bf in current term 2: Already voted for candidate e56f6424c0994b6f8ad2bbab85626f10 in this term.
I20260811 02:47:17.101549 2606 leader_election.cc:288] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [CANDIDATE]: Term 2 election: Requested vote from peers 2070708acfb041e58e41726a399e81be (127.21.236.190:41631), f60bba59016c4a489122f38ed2bf24bf (127.21.236.188:34191)
I20260811 02:47:17.101922 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.101687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.0.0.1:32972)
0811 02:47:17.101789 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.101848 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 02:47:17.101899 1745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e56f6424c0994b6f8ad2bbab85626f10" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f60bba59016c4a489122f38ed2bf24bf"
I20260811 02:47:17.102028 1745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:47:17.102414 1690 leader_election.cc:302] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f60bba59016c4a489122f38ed2bf24bf; no voters: 2070708acfb041e58e41726a399e81be, e56f6424c0994b6f8ad2bbab85626f10
I20260811 02:47:17.102725 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.102429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.0.0.1:32982)
0811 02:47:17.102613 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.102669 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260811 02:47:17.102844 1380 leader_election.cc:334] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2070708acfb041e58e41726a399e81be (127.21.236.190:41631): Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
I20260811 02:47:17.103309 1745 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56f6424c0994b6f8ad2bbab85626f10 in term 2.
I20260811 02:47:17.103431 2603 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:47:17.103631 1376 leader_election.cc:302] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56f6424c0994b6f8ad2bbab85626f10, f60bba59016c4a489122f38ed2bf24bf; no voters: 2070708acfb041e58e41726a399e81be
I20260811 02:47:17.103820 2606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:47:17.104045 2606 raft_consensus.cc:686] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [term 2 LEADER]: Becoming Leader. State: Replica: e56f6424c0994b6f8ad2bbab85626f10, State: Running, Role: LEADER
I20260811 02:47:17.104300 2606 consensus_queue.cc:237] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } }
I20260811 02:47:17.105683 2632 sys_catalog.cc:455] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e56f6424c0994b6f8ad2bbab85626f10. Latest consensus state: current_term: 2 leader_uuid: "e56f6424c0994b6f8ad2bbab85626f10" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } } }
I20260811 02:47:17.105835 2632 sys_catalog.cc:458] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:17.106192 2638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:17.108510 2638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:17.108902 2638 catalog_manager.cc:1259] Loaded cluster ID: fe3205df127e4363a58f17badb54d6dd
I20260811 02:47:17.108973 2638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:17.109529 2638 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:17.109913 2638 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: Loaded TSK: 0
I20260811 02:47:17.110582 2638 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:47:17.160894 1385 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:47:17.164389 1385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:47:17.167735 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.167431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.0.0.1:32998)
0811 02:47:17.167565 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.167650 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 02:47:17.167843 1745 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [term 2 FOLLOWER]: Refusing update from remote peer e56f6424c0994b6f8ad2bbab85626f10: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 02:47:17.168571 2606 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 02:47:17.168798 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.168335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.0.0.1:33008)
0811 02:47:17.168499 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.168557 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260811 02:47:17.168941 1380 consensus_peers.cc:487] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 -> Peer 2070708acfb041e58e41726a399e81be (127.21.236.190:41631): Couldn't send request to peer 2070708acfb041e58e41726a399e81be. Status: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 02:47:17.170318 2603 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e56f6424c0994b6f8ad2bbab85626f10. Latest consensus state: current_term: 2 leader_uuid: "e56f6424c0994b6f8ad2bbab85626f10" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } } }
I20260811 02:47:17.170500 2603 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:47:17.171586 2641 mvcc.cc:205] Tried to move back new op lower bound from 7317161726634053632 to 7317161726381686784. Current Snapshot: MvccSnapshot[applied={T|T < 7317161726634053632}]
I20260811 02:47:17.171752 2632 sys_catalog.cc:455] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "e56f6424c0994b6f8ad2bbab85626f10" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } } }
I20260811 02:47:17.171885 2632 sys_catalog.cc:458] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:17.172487 2603 sys_catalog.cc:455] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "e56f6424c0994b6f8ad2bbab85626f10" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2070708acfb041e58e41726a399e81be" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41631 } } peers { permanent_uuid: "e56f6424c0994b6f8ad2bbab85626f10" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 46833 } attrs { promote: false } } peers { permanent_uuid: "f60bba59016c4a489122f38ed2bf24bf" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 34191 } attrs { promote: false } } }
I20260811 02:47:17.172627 2603 sys_catalog.cc:458] T 00000000000000000000000000000000 P f60bba59016c4a489122f38ed2bf24bf [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:47:17.182804 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.177947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:46517 (local address 127.0.0.1:45898)
0811 02:47:17.178087 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.178103 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:17.178188 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:17.178980 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:17.178983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:17.179001 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:17.179143 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:17.179150 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.180197 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.180203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:17.181579 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:17.181590 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.182287 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.182291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:17.182408 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.182432 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:47:17.182531 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 02:47:17.182627 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":58}
I20260811 02:47:17.182816 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.178179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42169 (local address 127.0.0.1:58272)
0811 02:47:17.178413 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.178426 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:17.178518 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:17.178861 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:17.178864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:17.178883 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:17.179007 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:17.179015 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.180249 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.180252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:17.181452 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:17.181461 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.182369 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.182372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:17.182486 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.182512 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 02:47:17.182566 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 02:47:17.182661 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":90}
I20260811 02:47:17.183275 2652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.178312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58272 (local address 127.21.236.129:42169)
0811 02:47:17.178594 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:17.178598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:17.178611 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:47:17.178656 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:17.178658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:17.178726 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:17.179165 (+ 439us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:17.179295 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.180118 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.181579 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.182212 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.182935 (+ 723us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.182993 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:17.183031 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 02:47:17.183144 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":123,"threads_started":1}
I20260811 02:47:17.183348 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.178898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44264 (local address 127.21.236.130:33221)
0811 02:47:17.179217 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:17.179221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:17.179234 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:47:17.179288 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:17.179291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:17.179334 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:17.179425 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:17.179885 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.180573 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.181770 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.182614 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.182723 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.182771 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:17.183145 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 02:47:17.183218 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":76,"threads_started":1}
I20260811 02:47:17.183604 2654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.178741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:33221 (local address 127.0.0.1:44264)
0811 02:47:17.179027 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.179039 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:17.179153 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:17.179448 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:17.179451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:17.179467 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:17.179543 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:17.179549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.180682 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.180688 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:17.181654 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:17.181665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.182828 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.182831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:17.182919 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.182952 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 02:47:17.183020 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 02:47:17.183511 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":81,"threads_started":1}
I20260811 02:47:17.183758 2653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.178407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45898 (local address 127.21.236.131:46517)
0811 02:47:17.178694 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:17.178698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:17.178712 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:47:17.178763 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:17.178767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:17.178825 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:17.179256 (+ 431us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:17.179359 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.180071 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.181727 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.182163 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.182303 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.182358 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:17.183156 (+ 798us) server_negotiation.cc:299] Negotiation successful
0811 02:47:17.183251 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":82,"threads_started":1}
I20260811 02:47:17.183941 1917 tablet_service.cc:1467] Processing CreateTablet for tablet f19c5a7e1fb54929a521652d4517648c (DEFAULT_TABLE table=test-table [id=723ac96956f94921981a764c2158bae2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:17.184257 1917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f19c5a7e1fb54929a521652d4517648c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:17.186502 2657 tablet_bootstrap.cc:492] T f19c5a7e1fb54929a521652d4517648c P 1e0426b9d6594109ba2a0afbdf86107d: Bootstrap starting.
I20260811 02:47:17.186956 2366 tablet_service.cc:1467] Processing CreateTablet for tablet f19c5a7e1fb54929a521652d4517648c (DEFAULT_TABLE table=test-table [id=723ac96956f94921981a764c2158bae2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:17.186957 2147 tablet_service.cc:1467] Processing CreateTablet for tablet f19c5a7e1fb54929a521652d4517648c (DEFAULT_TABLE table=test-table [id=723ac96956f94921981a764c2158bae2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:17.187263 2147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f19c5a7e1fb54929a521652d4517648c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:17.187286 2366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f19c5a7e1fb54929a521652d4517648c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:17.187526 2657 tablet_bootstrap.cc:654] T f19c5a7e1fb54929a521652d4517648c P 1e0426b9d6594109ba2a0afbdf86107d: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:17.188855 2657 tablet_bootstrap.cc:492] T f19c5a7e1fb54929a521652d4517648c P 1e0426b9d6594109ba2a0afbdf86107d: No bootstrap required, opened a new log
I20260811 02:47:17.188957 2657 ts_tablet_manager.cc:1397] T f19c5a7e1fb54929a521652d4517648c P 1e0426b9d6594109ba2a0afbdf86107d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:47:17.189455 2657 raft_consensus.cc:348] T f19c5a7e1fb54929a521652d4517648c P 1e0426b9d6594109ba2a0afbdf86107d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } } peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } }
I20260811 02:47:17.189579 2657 raft_consensus.cc:374] T f19c5a7e1fb54929a521652d4517648c P 1e0426b9d6594109ba2a0afbdf86107d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:17.189625 2657 raft_consensus.cc:729] T f19c5a7e1fb54929a521652d4517648c P 1e0426b9d6594109ba2a0afbdf86107d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e0426b9d6594109ba2a0afbdf86107d, State: Initialized, Role: FOLLOWER
I20260811 02:47:17.189756 2657 consensus_queue.cc:260] T f19c5a7e1fb54929a521652d4517648c P 1e0426b9d6594109ba2a0afbdf86107d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } } peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } }
I20260811 02:47:17.190052 2659 tablet_bootstrap.cc:492] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088: Bootstrap starting.
I20260811 02:47:17.190105 2657 ts_tablet_manager.cc:1428] T f19c5a7e1fb54929a521652d4517648c P 1e0426b9d6594109ba2a0afbdf86107d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 02:47:17.190909 2660 tablet_bootstrap.cc:492] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8: Bootstrap starting.
I20260811 02:47:17.191680 2664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.190720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.129:38095)
0811 02:47:17.191513 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.191574 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":91,"threads_started":1}
I20260811 02:47:17.191882 2660 tablet_bootstrap.cc:654] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:17.192776 2664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.192333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.129:35351)
0811 02:47:17.192663 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.192714 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242}
I20260811 02:47:17.193161 2659 tablet_bootstrap.cc:654] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:17.193248 2660 tablet_bootstrap.cc:492] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8: No bootstrap required, opened a new log
I20260811 02:47:17.193337 2660 ts_tablet_manager.cc:1397] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 02:47:17.193356 2038 heartbeater.cc:503] Master 127.21.236.189:46833 was elected leader, sending a full tablet report...
I20260811 02:47:17.193791 2660 raft_consensus.cc:348] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } } peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } }
I20260811 02:47:17.193920 2660 raft_consensus.cc:374] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:17.193967 2660 raft_consensus.cc:729] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b645f57220a478cb25806d0c7da19e8, State: Initialized, Role: FOLLOWER
I20260811 02:47:17.194105 2660 consensus_queue.cc:260] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } } peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } }
I20260811 02:47:17.194466 2660 ts_tablet_manager.cc:1428] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 02:47:17.195776 2659 tablet_bootstrap.cc:492] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088: No bootstrap required, opened a new log
I20260811 02:47:17.195878 2659 ts_tablet_manager.cc:1397] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 02:47:17.196271 2669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.194887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.130:33297)
0811 02:47:17.196090 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.196150 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1129,"thread_start_us":102,"threads_started":1}
I20260811 02:47:17.196331 2659 raft_consensus.cc:348] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } } peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } }
I20260811 02:47:17.196641 2659 raft_consensus.cc:374] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:17.196695 2659 raft_consensus.cc:729] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70311aa80d1841af979f44135d73f088, State: Initialized, Role: FOLLOWER
I20260811 02:47:17.196836 2659 consensus_queue.cc:260] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } } peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } }
I20260811 02:47:17.196915 2669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.196657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.130:37791)
0811 02:47:17.196806 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.196853 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 02:47:17.197266 2659 ts_tablet_manager.cc:1428] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:47:17.197918 2671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.197516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.131:40909)
0811 02:47:17.197750 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.197810 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":110,"threads_started":1}
I20260811 02:47:17.198596 2671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.198377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.131:32915)
0811 02:47:17.198473 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.198531 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 02:47:17.198798 2491 heartbeater.cc:503] Master 127.21.236.189:46833 was elected leader, sending a full tablet report...
W20260811 02:47:17.212337 2252 tablet.cc:2369] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:47:17.221617 2039 tablet.cc:2369] T f19c5a7e1fb54929a521652d4517648c P 1e0426b9d6594109ba2a0afbdf86107d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:47:17.237306 2668 raft_consensus.cc:482] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:47:17.237468 2668 raft_consensus.cc:504] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } } peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } }
I20260811 02:47:17.238560 2668 leader_election.cc:288] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70311aa80d1841af979f44135d73f088 (127.21.236.131:46517), 1e0426b9d6594109ba2a0afbdf86107d (127.21.236.129:42169)
I20260811 02:47:17.243817 2653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.239092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:43553 (local address 127.21.236.131:46517)
0811 02:47:17.239225 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:17.239231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:17.239248 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:47:17.239345 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:17.239347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:17.239458 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:17.239545 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:17.239980 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.240857 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.242737 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.243267 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.243378 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.243418 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:17.243652 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 02:47:17.243714 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 02:47:17.243857 2669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.238858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:46517 (local address 127.21.236.130:43553)
0811 02:47:17.239037 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.239052 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:17.239225 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:17.239629 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:17.239632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:17.239657 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:17.239855 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:17.239866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.240972 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.240977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:17.242615 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:17.242626 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.243386 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.243390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:17.243508 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.243537 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 02:47:17.243625 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 02:47:17.243699 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":96,"spinlock_wait_cycles":10624}
I20260811 02:47:17.244508 2403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19c5a7e1fb54929a521652d4517648c" candidate_uuid: "0b645f57220a478cb25806d0c7da19e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70311aa80d1841af979f44135d73f088" is_pre_election: true
I20260811 02:47:17.244639 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.239201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42169 (local address 127.21.236.130:38217)
0811 02:47:17.239551 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.239567 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:17.239752 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:17.240058 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:17.240061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:17.240080 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:17.240195 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:17.240203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.241108 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.241111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:17.242615 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:17.242626 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.244136 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.244142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:17.244234 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.244252 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 02:47:17.244380 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 02:47:17.244460 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":66,"spinlock_wait_cycles":2048,"thread_start_us":140,"threads_started":1}
I20260811 02:47:17.244639 2652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.239351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:38217 (local address 127.21.236.129:42169)
0811 02:47:17.239537 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:17.239542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:17.239650 (+ 108us) server_negotiation.cc:407] Connection header received
0811 02:47:17.239886 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:17.239890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:17.239960 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:17.240049 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:17.240298 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.240951 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.243183 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.243859 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.243988 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.244053 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:17.244405 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 02:47:17.244495 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:47:17.244792 2403 raft_consensus.cc:2454] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b645f57220a478cb25806d0c7da19e8 in term 0.
I20260811 02:47:17.245472 1963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19c5a7e1fb54929a521652d4517648c" candidate_uuid: "0b645f57220a478cb25806d0c7da19e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e0426b9d6594109ba2a0afbdf86107d" is_pre_election: true
I20260811 02:47:17.245657 1963 raft_consensus.cc:2454] T f19c5a7e1fb54929a521652d4517648c P 1e0426b9d6594109ba2a0afbdf86107d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b645f57220a478cb25806d0c7da19e8 in term 0.
I20260811 02:47:17.245929 2078 leader_election.cc:302] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b645f57220a478cb25806d0c7da19e8, 70311aa80d1841af979f44135d73f088; no voters:
I20260811 02:47:17.246223 2668 raft_consensus.cc:2781] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:47:17.246295 2668 raft_consensus.cc:482] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:47:17.246340 2668 raft_consensus.cc:3037] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:17.247563 2668 raft_consensus.cc:504] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } } peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } }
I20260811 02:47:17.247936 2668 leader_election.cc:288] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [CANDIDATE]: Term 1 election: Requested vote from peers 70311aa80d1841af979f44135d73f088 (127.21.236.131:46517), 1e0426b9d6594109ba2a0afbdf86107d (127.21.236.129:42169)
I20260811 02:47:17.248312 2403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19c5a7e1fb54929a521652d4517648c" candidate_uuid: "0b645f57220a478cb25806d0c7da19e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70311aa80d1841af979f44135d73f088"
I20260811 02:47:17.248440 2403 raft_consensus.cc:3037] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:17.248389 1963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f19c5a7e1fb54929a521652d4517648c" candidate_uuid: "0b645f57220a478cb25806d0c7da19e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e0426b9d6594109ba2a0afbdf86107d"
I20260811 02:47:17.248667 1963 raft_consensus.cc:3037] T f19c5a7e1fb54929a521652d4517648c P 1e0426b9d6594109ba2a0afbdf86107d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:17.249536 2403 raft_consensus.cc:2454] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b645f57220a478cb25806d0c7da19e8 in term 1.
I20260811 02:47:17.249709 1963 raft_consensus.cc:2454] T f19c5a7e1fb54929a521652d4517648c P 1e0426b9d6594109ba2a0afbdf86107d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b645f57220a478cb25806d0c7da19e8 in term 1.
I20260811 02:47:17.249876 2078 leader_election.cc:302] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b645f57220a478cb25806d0c7da19e8, 70311aa80d1841af979f44135d73f088; no voters:
I20260811 02:47:17.250074 2668 raft_consensus.cc:2781] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:17.250319 2668 raft_consensus.cc:686] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [term 1 LEADER]: Becoming Leader. State: Replica: 0b645f57220a478cb25806d0c7da19e8, State: Running, Role: LEADER
I20260811 02:47:17.250618 2668 consensus_queue.cc:237] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } } peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } }
I20260811 02:47:17.250747 2251 heartbeater.cc:503] Master 127.21.236.189:46833 was elected leader, sending a full tablet report...
I20260811 02:47:17.251202 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.250890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.130:46543)
0811 02:47:17.251041 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.251128 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 02:47:17.253310 1385 catalog_manager.cc:5495] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b645f57220a478cb25806d0c7da19e8 (127.21.236.130). New cstate: current_term: 1 leader_uuid: "0b645f57220a478cb25806d0c7da19e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b645f57220a478cb25806d0c7da19e8" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33221 } health_report { overall_health: HEALTHY } } }
I20260811 02:47:17.283998 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.280407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44278 (local address 127.21.236.130:33221)
0811 02:47:17.280580 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:17.280584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:17.280597 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:47:17.280675 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:17.280677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:17.280733 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:17.280825 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:47:17.281044 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.281600 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.282393 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:17.282680 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:17.282761 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:17.283016 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:47:17.283580 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:47:17.283717 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:17.283775 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 02:47:17.283827 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:47:17.287115 1963 raft_consensus.cc:1261] T f19c5a7e1fb54929a521652d4517648c P 1e0426b9d6594109ba2a0afbdf86107d [term 1 FOLLOWER]: Refusing update from remote peer 0b645f57220a478cb25806d0c7da19e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:17.287492 2403 raft_consensus.cc:1261] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088 [term 1 FOLLOWER]: Refusing update from remote peer 0b645f57220a478cb25806d0c7da19e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:17.287737 2668 consensus_queue.cc:1035] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e0426b9d6594109ba2a0afbdf86107d" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:17.288125 2675 consensus_queue.cc:1035] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70311aa80d1841af979f44135d73f088" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 46517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:17.289549 2664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.289204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.129:49207)
0811 02:47:17.289343 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.289463 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 02:47:17.291783 2671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.291491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.131:49633)
0811 02:47:17.291643 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.291699 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 02:47:17.294577 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.294337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.130:42503)
0811 02:47:17.294439 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.294500 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 02:47:17.295691 2671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.295422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.131:41563)
0811 02:47:17.295560 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.295612 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 02:47:17.295894 2664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.294617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.129:45009)
0811 02:47:17.295731 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.295819 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1037}
I20260811 02:47:17.296195 2674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.295103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41631 (local address 127.21.236.130:58655)
0811 02:47:17.296051 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.296112 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":868}
I20260811 02:47:17.296458 2680 mvcc.cc:205] Tried to move back new op lower bound from 7317161727122468864 to 7317161726981988352. Current Snapshot: MvccSnapshot[applied={T|T < 7317161727122468864}]
I20260811 02:47:17.296739 2683 mvcc.cc:205] Tried to move back new op lower bound from 7317161727122468864 to 7317161726981988352. Current Snapshot: MvccSnapshot[applied={T|T < 7317161727122468864}]
I20260811 02:47:17.304812 1384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:17.305025 1384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:17.309165 1384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e56f6424c0994b6f8ad2bbab85626f10: Sending DeleteTablet for 3 replicas of tablet f19c5a7e1fb54929a521652d4517648c
I20260811 02:47:17.310011 2147 tablet_service.cc:1514] Processing DeleteTablet for tablet f19c5a7e1fb54929a521652d4517648c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:17 UTC) from {username='oryx', principal='oryx/127.21.236.189@KRBTEST.COM'} at 127.0.0.1:44264
I20260811 02:47:17.310217 1917 tablet_service.cc:1514] Processing DeleteTablet for tablet f19c5a7e1fb54929a521652d4517648c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:17 UTC) from {username='oryx', principal='oryx/127.21.236.189@KRBTEST.COM'} at 127.0.0.1:58272
I20260811 02:47:17.310552 2700 tablet_replica.cc:330] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8: stopping tablet replica
I20260811 02:47:17.310545 2366 tablet_service.cc:1514] Processing DeleteTablet for tablet f19c5a7e1fb54929a521652d4517648c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:17 UTC) from {username='oryx', principal='oryx/127.21.236.189@KRBTEST.COM'} at 127.0.0.1:45898
I20260811 02:47:17.310696 2700 raft_consensus.cc:2229] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:47:17.310925 2704 tablet_replica.cc:330] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088: stopping tablet replica
I20260811 02:47:17.311102 2700 raft_consensus.cc:2258] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:17.311144 2704 raft_consensus.cc:2229] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:17.311416 2704 raft_consensus.cc:2258] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:17.311607 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 1640
I20260811 02:47:17.315135 2700 ts_tablet_manager.cc:1905] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:17.323787 2700 ts_tablet_manager.cc:1918] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:17.323917 2700 log.cc:1196] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/f19c5a7e1fb54929a521652d4517648c
I20260811 02:47:17.324273 2700 ts_tablet_manager.cc:1939] T f19c5a7e1fb54929a521652d4517648c P 0b645f57220a478cb25806d0c7da19e8: Deleting consensus metadata
I20260811 02:47:17.324353 2704 ts_tablet_manager.cc:1905] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:17.325212 1376 catalog_manager.cc:4841] TS 0b645f57220a478cb25806d0c7da19e8 (127.21.236.130:33221): tablet f19c5a7e1fb54929a521652d4517648c (table test-table [id=723ac96956f94921981a764c2158bae2]) successfully deleted
I20260811 02:47:17.326722 2704 ts_tablet_manager.cc:1918] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:17.326828 2704 log.cc:1196] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/f19c5a7e1fb54929a521652d4517648c
I20260811 02:47:17.327181 2704 ts_tablet_manager.cc:1939] T f19c5a7e1fb54929a521652d4517648c P 70311aa80d1841af979f44135d73f088: Deleting consensus metadata
I20260811 02:47:17.327875 1374 catalog_manager.cc:4841] TS 70311aa80d1841af979f44135d73f088 (127.21.236.131:46517): tablet f19c5a7e1fb54929a521652d4517648c (table test-table [id=723ac96956f94921981a764c2158bae2]) successfully deleted
I20260811 02:47:17.332958 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 2051
I20260811 02:47:17.336182 2654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:17.333117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42169 (local address 127.0.0.1:58276)
0811 02:47:17.335987 (+ 2870us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:17.336066 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:42169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2736}
W20260811 02:47:17.338430 1374 catalog_manager.cc:4584] TS 1e0426b9d6594109ba2a0afbdf86107d (127.21.236.129:42169): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f19c5a7e1fb54929a521652d4517648c: Network error: Client connection negotiation failed: client connection to 127.21.236.129:42169: connect: Connection refused (error 111)
I20260811 02:47:17.349746 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 2259
I20260811 02:47:17.363610 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 1173
I20260811 02:47:17.375437 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 1352
2026-08-11T02:47:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5829 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:47:17 dist-test-slave-n326 krb5kdc[2707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:47:17 dist-test-slave-n326 krb5kdc[2707](info): set up 2 sockets
Aug 11 02:47:17 dist-test-slave-n326 krb5kdc[2707](info): commencing operation
krb5kdc: starting...
W20260811 02:47:18.673748 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:47:18 dist-test-slave-n326 krb5kdc[2707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:47:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:18Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.190.keytab.
Entry for principal oryx/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.190.keytab.
I20260811 02:47:18.811802 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:37597
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.190.keytab
--principal=oryx/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:37597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:18.929557 2729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:18.929857 2729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:18.929939 2729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:18.934021 2729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:18.934118 2729 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 02:47:18.934149 2729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:18.934191 2729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:18.934230 2729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:18.934266 2729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:47:18.939677 2729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:37597
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.190.keytab
--principal=oryx/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:37597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:18.940826 2729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:18.941946 2729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:18.951138 2739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:18.951125 2743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:18.951094 2737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:18.951460 2729 server_base.cc:1034] running on GCE node
I20260811 02:47:18.952124 2729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:18.953289 2729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:18.954535 2729 hybrid_clock.cc:648] HybridClock initialized: now 1786416438954521 us; error 34 us; skew 500 ppm
Aug 11 02:47:18 dist-test-slave-n326 krb5kdc[2707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416438, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:18.961519 2729 init.cc:377] Logged in from keytab as oryx/127.21.236.190@KRBTEST.COM (short username oryx)
I20260811 02:47:18.962606 2729 webserver.cc:460] Webserver started at http://127.21.236.190:45777/ using document root <none> and password file <none>
I20260811 02:47:18.963248 2729 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:18.963322 2729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:18.963550 2729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:18.965677 2729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "de51c0fa70c344e0a22dab0e6c92f2e3"
format_stamp: "Formatted at 2026-08-11 02:47:18 on dist-test-slave-n326"
server_key: "d4092389c6ed77d17ff8ea948ac1b918"
server_key_iv: "767832aaeb5d13f676eba2396291d206"
server_key_version: "encryptionkey@0"
I20260811 02:47:18.966213 2729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "de51c0fa70c344e0a22dab0e6c92f2e3"
format_stamp: "Formatted at 2026-08-11 02:47:18 on dist-test-slave-n326"
server_key: "d4092389c6ed77d17ff8ea948ac1b918"
server_key_iv: "767832aaeb5d13f676eba2396291d206"
server_key_version: "encryptionkey@0"
I20260811 02:47:18.970203 2729 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:47:18.972738 2755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:18.974025 2729 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:47:18.974162 2729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "de51c0fa70c344e0a22dab0e6c92f2e3"
format_stamp: "Formatted at 2026-08-11 02:47:18 on dist-test-slave-n326"
server_key: "d4092389c6ed77d17ff8ea948ac1b918"
server_key_iv: "767832aaeb5d13f676eba2396291d206"
server_key_version: "encryptionkey@0"
I20260811 02:47:18.974277 2729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:18.988489 2729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:18.992014 2729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:18.992198 2729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:18.999843 2729 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:37597
I20260811 02:47:18.999913 2868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:37597 every 8 connection(s)
I20260811 02:47:19.000849 2729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:19.003660 2869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:19.009840 2869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3: Bootstrap starting.
I20260811 02:47:19.010826 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 2729
I20260811 02:47:19.010985 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:47:19.011402 22450 external_mini_cluster.cc:1428] Setting key fe2309a3ecc75dfb55d2c0bea0eb9332
I20260811 02:47:19.013281 2869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:19.014148 2869 log.cc:824] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:19.017035 2869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3: No bootstrap required, opened a new log
I20260811 02:47:19.021742 2869 raft_consensus.cc:348] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de51c0fa70c344e0a22dab0e6c92f2e3" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 37597 } }
I20260811 02:47:19.022029 2869 raft_consensus.cc:374] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:19.022119 2869 raft_consensus.cc:729] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de51c0fa70c344e0a22dab0e6c92f2e3, State: Initialized, Role: FOLLOWER
Aug 11 02:47:19 dist-test-slave-n326 krb5kdc[2707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
I20260811 02:47:19.022617 2869 consensus_queue.cc:260] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de51c0fa70c344e0a22dab0e6c92f2e3" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 37597 } }
I20260811 02:47:19.022794 2869 raft_consensus.cc:388] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:19.022886 2869 raft_consensus.cc:482] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:19.023012 2869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:19.024390 2869 raft_consensus.cc:504] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de51c0fa70c344e0a22dab0e6c92f2e3" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 37597 } }
I20260811 02:47:19.024782 2869 leader_election.cc:302] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de51c0fa70c344e0a22dab0e6c92f2e3; no voters:
I20260811 02:47:19.025197 2869 leader_election.cc:288] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:47:19.025401 2878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:19.025720 2878 raft_consensus.cc:686] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [term 1 LEADER]: Becoming Leader. State: Replica: de51c0fa70c344e0a22dab0e6c92f2e3, State: Running, Role: LEADER
I20260811 02:47:19.026062 2878 consensus_queue.cc:237] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de51c0fa70c344e0a22dab0e6c92f2e3" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 37597 } }
I20260811 02:47:19.026697 2869 sys_catalog.cc:564] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:19.027902 2880 sys_catalog.cc:455] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de51c0fa70c344e0a22dab0e6c92f2e3. Latest consensus state: current_term: 1 leader_uuid: "de51c0fa70c344e0a22dab0e6c92f2e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de51c0fa70c344e0a22dab0e6c92f2e3" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 37597 } } }
I20260811 02:47:19.027904 2879 sys_catalog.cc:455] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de51c0fa70c344e0a22dab0e6c92f2e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de51c0fa70c344e0a22dab0e6c92f2e3" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 37597 } } }
I20260811 02:47:19.028069 2879 sys_catalog.cc:458] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:19.028069 2880 sys_catalog.cc:458] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:19.029523 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.012788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46228 (local address 127.21.236.190:37597)
0811 02:47:19.013328 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:19.013344 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:19.013423 (+ 79us) server_negotiation.cc:407] Connection header received
0811 02:47:19.014245 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:19.014267 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:19.014709 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:19.015155 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:19.016212 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.017760 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.019151 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.019734 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.019869 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.022984 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:19.022991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:19.023011 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:19.023049 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:19.025263 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.025831 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.025837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.025842 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.025981 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.026312 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.026315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.026317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.026852 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:19.027859 (+ 1007us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:19.028021 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 02:47:19.028338 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":165,"threads_started":1}
W20260811 02:47:19.031874 2912 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:47:19.031953 2912 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:47:19.032083 2897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:19.032825 2897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:19.038398 2897 catalog_manager.cc:1347] Generated new cluster ID: 8104ea84030a4390a562af7f29a0d077
I20260811 02:47:19.038482 2897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:19.050663 2897 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:47:19.051738 2897 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:19.066219 2897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3: Generated new TSK 0
I20260811 02:47:19.067071 2897 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.129.keytab.
Entry for principal oryx/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.129.keytab.
I20260811 02:47:19.110174 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:37597
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.129.keytab
--principal=oryx/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:19.230875 2923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:19.231246 2923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:19.231300 2923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:19.235450 2923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:19.235522 2923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:19.235562 2923 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:47:19.235596 2923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:47:19.240980 2923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.129.keytab
--principal=oryx/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.236.190:37597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:19.242056 2923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:19.243080 2923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:19.252929 2932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:19.252929 2930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:19.253060 2936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:19.253297 2923 server_base.cc:1034] running on GCE node
I20260811 02:47:19.253746 2923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:19.254432 2923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:19.255621 2923 hybrid_clock.cc:648] HybridClock initialized: now 1786416439255596 us; error 37 us; skew 500 ppm
Aug 11 02:47:19 dist-test-slave-n326 krb5kdc[2707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416439, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:19.258850 2923 init.cc:377] Logged in from keytab as oryx/127.21.236.129@KRBTEST.COM (short username oryx)
I20260811 02:47:19.259934 2923 webserver.cc:460] Webserver started at http://127.21.236.129:38351/ using document root <none> and password file <none>
I20260811 02:47:19.260566 2923 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:19.260643 2923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:19.260880 2923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:19.263217 2923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "e65c1aaacde848ebaf5ce2df1b01e025"
format_stamp: "Formatted at 2026-08-11 02:47:19 on dist-test-slave-n326"
server_key: "39e4d555a1dd2cd7c4b0d1e6a3f10de2"
server_key_iv: "3130e196b3254a297d43a5b0e6e5e91e"
server_key_version: "encryptionkey@0"
I20260811 02:47:19.263846 2923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "e65c1aaacde848ebaf5ce2df1b01e025"
format_stamp: "Formatted at 2026-08-11 02:47:19 on dist-test-slave-n326"
server_key: "39e4d555a1dd2cd7c4b0d1e6a3f10de2"
server_key_iv: "3130e196b3254a297d43a5b0e6e5e91e"
server_key_version: "encryptionkey@0"
I20260811 02:47:19.267968 2923 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:47:19.270673 2951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:19.272146 2923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:47:19.272305 2923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "e65c1aaacde848ebaf5ce2df1b01e025"
format_stamp: "Formatted at 2026-08-11 02:47:19 on dist-test-slave-n326"
server_key: "39e4d555a1dd2cd7c4b0d1e6a3f10de2"
server_key_iv: "3130e196b3254a297d43a5b0e6e5e91e"
server_key_version: "encryptionkey@0"
I20260811 02:47:19.272437 2923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:19.298715 2923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:19.302261 2923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:19.302506 2923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:19.304284 2923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:19.304378 2923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:19.304479 2923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:19.304543 2923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:47:19 dist-test-slave-n326 krb5kdc[2707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416439, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
I20260811 02:47:19.317173 2923 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:39753
I20260811 02:47:19.317968 3160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:39753 every 8 connection(s)
I20260811 02:47:19.318334 2923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:47:19.320387 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.305120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:45933 (local address 127.21.236.190:37597)
0811 02:47:19.305250 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:19.305254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:19.306495 (+ 1241us) server_negotiation.cc:407] Connection header received
0811 02:47:19.307715 (+ 1220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:19.307720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:19.307784 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:19.307881 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:19.309566 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.310590 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.312025 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.312406 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.312556 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.315938 (+ 3382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:19.315943 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:19.315946 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:19.315983 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:19.317753 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.318457 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.318461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.318463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.318561 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.319023 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.319028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.319033 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.319353 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:19.319485 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:19.320043 (+ 558us) server_negotiation.cc:299] Negotiation successful
0811 02:47:19.320200 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 02:47:19.321169 2969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.305713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:37597 (local address 127.21.236.129:45933)
0811 02:47:19.306310 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:19.306356 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:19.307483 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:19.308173 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:19.308182 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:19.308758 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:19.309362 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.309378 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.310756 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.310760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.311878 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.311890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.312605 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.312610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.312837 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.313602 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:19.313627 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:19.315672 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:19.317884 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:19.317893 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:19.317911 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:19.318305 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:19.318669 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:19.318675 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:19.318679 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:19.318887 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:19.319499 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:19.319509 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:19.319773 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 02:47:19.319974 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 02:47:19.320319 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":147,"threads_started":1}
I20260811 02:47:19.323968 3161 heartbeater.cc:347] Connected to a master server at 127.21.236.190:37597
I20260811 02:47:19.324162 3161 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:19.324777 3161 heartbeater.cc:511] Master 127.21.236.190:37597 requested a full tablet report, sending...
I20260811 02:47:19.326666 2783 ts_manager.cc:194] Registered new tserver with Master: e65c1aaacde848ebaf5ce2df1b01e025 (127.21.236.129:39753)
I20260811 02:47:19.328457 2783 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:45933
I20260811 02:47:19.328615 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 2923
I20260811 02:47:19.328759 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:47:19.329155 22450 external_mini_cluster.cc:1428] Setting key 13ceff7f8bf706fdee9afbcc89db27c8
I20260811 02:47:19.335321 3171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.326353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:37597 (local address 127.21.236.129:58275)
0811 02:47:19.326714 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:19.326731 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:19.326899 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:19.327323 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:19.327326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:19.327605 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:19.327745 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.327753 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.328722 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.328726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.329500 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.329512 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.330217 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.330221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.330311 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.330892 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:19.330907 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:19.331397 (+ 490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:19.333582 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:19.333586 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:19.333588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:19.333879 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:19.334216 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:19.334222 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:19.334227 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:19.334338 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:19.334818 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:19.334820 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:19.334902 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 02:47:19.334988 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 02:47:19.335168 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":108,"threads_started":1}
I20260811 02:47:19.335500 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.326419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:58275 (local address 127.21.236.190:37597)
0811 02:47:19.326554 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:19.326558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:19.326816 (+ 258us) server_negotiation.cc:407] Connection header received
0811 02:47:19.327046 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:19.327049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:19.327192 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:19.327299 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:19.327879 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.328543 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.329642 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.329943 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.330030 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.331555 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:19.331562 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:19.331569 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:19.331612 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:19.333451 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.334013 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.334016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.334025 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.334110 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.334448 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.334450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.334451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.334713 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:19.334821 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:19.335133 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 02:47:19.335286 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":55}
WARNING: no policy specified for oryx/127.21.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.130.keytab.
Entry for principal oryx/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.130.keytab.
I20260811 02:47:19.363250 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:37597
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.130.keytab
--principal=oryx/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:19.484519 3177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:19.484793 3177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:19.484838 3177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:19.488775 3177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:19.488828 3177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:19.488865 3177 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:47:19.488897 3177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:47:19.494086 3177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.130.keytab
--principal=oryx/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.236.190:37597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:19.495216 3177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:19.496193 3177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:19.506058 3186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:19.506065 3184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:19.506156 3177 server_base.cc:1034] running on GCE node
W20260811 02:47:19.506065 3188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:19.506744 3177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:19.507380 3177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:19.508550 3177 hybrid_clock.cc:648] HybridClock initialized: now 1786416439508530 us; error 34 us; skew 500 ppm
Aug 11 02:47:19 dist-test-slave-n326 krb5kdc[2707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416439, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:19.511373 3177 init.cc:377] Logged in from keytab as oryx/127.21.236.130@KRBTEST.COM (short username oryx)
I20260811 02:47:19.512230 3177 webserver.cc:460] Webserver started at http://127.21.236.130:43765/ using document root <none> and password file <none>
I20260811 02:47:19.512810 3177 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:19.512872 3177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:19.513089 3177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:19.515151 3177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "75f5fd4a5c33444da5694e653491cdd1"
format_stamp: "Formatted at 2026-08-11 02:47:19 on dist-test-slave-n326"
server_key: "fd4e533f9880e762f4dc9885a8adc4b3"
server_key_iv: "9bdb48a31b7c4bb5413993d03bb74d37"
server_key_version: "encryptionkey@0"
I20260811 02:47:19.515724 3177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "75f5fd4a5c33444da5694e653491cdd1"
format_stamp: "Formatted at 2026-08-11 02:47:19 on dist-test-slave-n326"
server_key: "fd4e533f9880e762f4dc9885a8adc4b3"
server_key_iv: "9bdb48a31b7c4bb5413993d03bb74d37"
server_key_version: "encryptionkey@0"
I20260811 02:47:19.519564 3177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 02:47:19.521977 3206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:19.523232 3177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:47:19.523362 3177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "75f5fd4a5c33444da5694e653491cdd1"
format_stamp: "Formatted at 2026-08-11 02:47:19 on dist-test-slave-n326"
server_key: "fd4e533f9880e762f4dc9885a8adc4b3"
server_key_iv: "9bdb48a31b7c4bb5413993d03bb74d37"
server_key_version: "encryptionkey@0"
I20260811 02:47:19.523531 3177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:19.536015 3177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:19.539270 3177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:19.539475 3177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:19.540879 3177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:19.540962 3177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:19.541047 3177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:19.541095 3177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:47:19 dist-test-slave-n326 krb5kdc[2707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416439, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
I20260811 02:47:19.551872 3177 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:43023
I20260811 02:47:19.552083 3418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:43023 every 8 connection(s)
I20260811 02:47:19.552928 3177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:47:19.554656 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.541744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:51559 (local address 127.21.236.190:37597)
0811 02:47:19.541913 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:19.541919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:19.542756 (+ 837us) server_negotiation.cc:407] Connection header received
0811 02:47:19.543797 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:19.543805 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:19.543875 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:19.543995 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:19.545615 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.546503 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.547950 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.548179 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.548295 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.550849 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:19.550853 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:19.550856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:19.550884 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:19.552513 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.553125 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.553129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.553131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.553236 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.553667 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.553670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.553672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.553905 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:19.553986 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:19.554417 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 02:47:19.554513 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:47:19.555461 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.542133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:37597 (local address 127.21.236.130:51559)
0811 02:47:19.542593 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:19.542641 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:19.543550 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:19.544284 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:19.544295 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:19.544802 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:19.545440 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.545459 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.546645 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.546648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.547826 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.547839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.548344 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.548348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.548515 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.549093 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:19.549109 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:19.550661 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:19.552633 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:19.552639 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:19.552651 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:19.553001 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:19.553344 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:19.553349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:19.553351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:19.553512 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:19.554009 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:19.554014 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:19.554212 (+ 198us) client_negotiation.cc:769] Sending connection context
0811 02:47:19.554363 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 02:47:19.554594 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":124,"threads_started":1}
I20260811 02:47:19.556551 3419 heartbeater.cc:347] Connected to a master server at 127.21.236.190:37597
I20260811 02:47:19.556716 3419 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:19.557207 3419 heartbeater.cc:511] Master 127.21.236.190:37597 requested a full tablet report, sending...
I20260811 02:47:19.558295 2783 ts_manager.cc:194] Registered new tserver with Master: 75f5fd4a5c33444da5694e653491cdd1 (127.21.236.130:43023)
I20260811 02:47:19.559087 2783 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:51559
I20260811 02:47:19.561386 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 3177
I20260811 02:47:19.561530 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:47:19.561870 22450 external_mini_cluster.cc:1428] Setting key d7647915b2aacd48def6b2af8287ee99
I20260811 02:47:19.566107 3433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.558587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:37597 (local address 127.21.236.130:44865)
0811 02:47:19.558863 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:19.558884 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:19.558962 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:19.559261 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:19.559264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:19.559499 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:19.559606 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.559612 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.560554 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.560557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.561160 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.561172 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.561700 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.561704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.561809 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.562255 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:19.562269 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:19.562664 (+ 395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:19.564441 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:19.564447 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:19.564452 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:19.564721 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:19.564984 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:19.564989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:19.564994 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:19.565105 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:19.565613 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:19.565615 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:19.565688 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 02:47:19.565771 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 02:47:19.565968 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
I20260811 02:47:19.566365 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.558682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:44865 (local address 127.21.236.190:37597)
0811 02:47:19.558805 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:19.558809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:19.558951 (+ 142us) server_negotiation.cc:407] Connection header received
0811 02:47:19.559040 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:19.559043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:19.559110 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:19.559191 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:19.559701 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.560410 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.561279 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.561570 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.561687 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.562786 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:19.562789 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:19.562792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:19.562813 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:19.564322 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.564820 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.564822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.564824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.564894 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.565201 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.565203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.565204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.565443 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:19.565557 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:19.565873 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 02:47:19.566012 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for oryx/127.21.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.131.keytab.
Entry for principal oryx/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.131.keytab.
I20260811 02:47:19.590610 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:37597
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.131.keytab
--principal=oryx/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:19.704797 3441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:19.705037 3441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:19.705080 3441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:19.708884 3441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:19.708932 3441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:19.708967 3441 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 02:47:19.708998 3441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:47:19.714145 3441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.131.keytab
--principal=oryx/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.236.190:37597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:19.715111 3441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:19.716050 3441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:19.725940 3452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:19.725952 3450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:19.725952 3448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:19.726166 3441 server_base.cc:1034] running on GCE node
I20260811 02:47:19.726702 3441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:19.727537 3441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:19.728719 3441 hybrid_clock.cc:648] HybridClock initialized: now 1786416439728692 us; error 48 us; skew 500 ppm
Aug 11 02:47:19 dist-test-slave-n326 krb5kdc[2707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416439, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:19.731474 3441 init.cc:377] Logged in from keytab as oryx/127.21.236.131@KRBTEST.COM (short username oryx)
I20260811 02:47:19.732372 3441 webserver.cc:460] Webserver started at http://127.21.236.131:38707/ using document root <none> and password file <none>
I20260811 02:47:19.732960 3441 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:19.733062 3441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:19.733290 3441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:19.735388 3441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "e6c99522e0164493ab63bd6137c8e85f"
format_stamp: "Formatted at 2026-08-11 02:47:19 on dist-test-slave-n326"
server_key: "5481baea3d85e08c047c4eecebeec123"
server_key_iv: "c53082619bf1e2f4f56339a14d483ed6"
server_key_version: "encryptionkey@0"
I20260811 02:47:19.735951 3441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "e6c99522e0164493ab63bd6137c8e85f"
format_stamp: "Formatted at 2026-08-11 02:47:19 on dist-test-slave-n326"
server_key: "5481baea3d85e08c047c4eecebeec123"
server_key_iv: "c53082619bf1e2f4f56339a14d483ed6"
server_key_version: "encryptionkey@0"
I20260811 02:47:19.739653 3441 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:47:19.742028 3464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:19.743300 3441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:47:19.743434 3441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "e6c99522e0164493ab63bd6137c8e85f"
format_stamp: "Formatted at 2026-08-11 02:47:19 on dist-test-slave-n326"
server_key: "5481baea3d85e08c047c4eecebeec123"
server_key_iv: "c53082619bf1e2f4f56339a14d483ed6"
server_key_version: "encryptionkey@0"
I20260811 02:47:19.743548 3441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:19.766289 3441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:19.769281 3441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:19.769490 3441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:19.771169 3441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:19.771237 3441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:19.771304 3441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:19.771354 3441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 02:47:19 dist-test-slave-n326 krb5kdc[2707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416439, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
I20260811 02:47:19.783219 3441 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:40905
I20260811 02:47:19.783288 3659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:40905 every 8 connection(s)
I20260811 02:47:19.784349 3441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:47:19.785719 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.771935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:33857 (local address 127.21.236.190:37597)
0811 02:47:19.772102 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:19.772107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:19.772920 (+ 813us) server_negotiation.cc:407] Connection header received
0811 02:47:19.773863 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:19.773867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:19.773932 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:19.774030 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:19.775466 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.776067 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.777436 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.777653 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.777782 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.781715 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:19.781719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:19.781722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:19.781751 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:19.783516 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.784070 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.784074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.784076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.784172 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.784500 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.784503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.784505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.784752 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:19.784865 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:19.785476 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 02:47:19.785598 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:47:19.786513 3479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.772318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:37597 (local address 127.21.236.131:33857)
0811 02:47:19.772750 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:19.772800 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:19.773651 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:19.774205 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:19.774212 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:19.774705 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:19.775303 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.775322 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.776194 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.776198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.777302 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.777313 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.777808 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.777813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.778036 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.778705 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:19.778729 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:19.781493 (+ 2764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:19.783643 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:19.783647 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:19.783678 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:19.783948 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:19.784269 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:19.784271 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:19.784273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:19.784399 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:19.784919 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:19.784925 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:19.785249 (+ 324us) client_negotiation.cc:769] Sending connection context
0811 02:47:19.785426 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 02:47:19.785665 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":125,"threads_started":1}
I20260811 02:47:19.787698 3660 heartbeater.cc:347] Connected to a master server at 127.21.236.190:37597
I20260811 02:47:19.787860 3660 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:19.788352 3660 heartbeater.cc:511] Master 127.21.236.190:37597 requested a full tablet report, sending...
I20260811 02:47:19.789441 2784 ts_manager.cc:194] Registered new tserver with Master: e6c99522e0164493ab63bd6137c8e85f (127.21.236.131:40905)
I20260811 02:47:19.790256 2784 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:33857
I20260811 02:47:19.792533 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 3441
I20260811 02:47:19.792662 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:47:19.793005 22450 external_mini_cluster.cc:1428] Setting key 7eab90c017afcaa62e5664c6c1c4eb09
I20260811 02:47:19.795337 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:47:19.797343 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.789754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:37597 (local address 127.21.236.131:44443)
0811 02:47:19.790051 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:19.790064 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:19.790153 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:19.790376 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:19.790379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:19.790595 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:19.790687 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.790692 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.791552 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.791555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.792351 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.792360 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.792963 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.792966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.793035 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.793620 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:19.793631 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:19.794083 (+ 452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:19.795959 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:19.795962 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:19.795963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:19.796151 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:19.796445 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:19.796447 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:19.796448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:19.796520 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:19.796953 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:19.796955 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:19.797027 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 02:47:19.797114 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 02:47:19.797212 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":94,"threads_started":1}
I20260811 02:47:19.797562 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.789838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:44443 (local address 127.21.236.190:37597)
0811 02:47:19.789944 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:19.789948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:19.790131 (+ 183us) server_negotiation.cc:407] Connection header received
0811 02:47:19.790229 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:19.790231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:19.790266 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:19.790329 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:19.790793 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.791422 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.792474 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.792852 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.792964 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.794215 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:19.794219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:19.794221 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:19.794251 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:19.795845 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.796253 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.796259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.796263 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.796353 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.796608 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.796614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.796619 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.796852 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:19.796955 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:19.797195 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 02:47:19.797323 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 02:47:19 dist-test-slave-n326 krb5kdc[2707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416439, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 02:47:19 dist-test-slave-n326 krb5kdc[2707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:19 dist-test-slave-n326 krb5kdc[2707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 02:47:19.822491 2875 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:19.814671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46240 (local address 127.21.236.190:37597)
0811 02:47:19.814799 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:19.814803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:19.814816 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:47:19.814857 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:19.814860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:19.814898 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:19.814968 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:19.816080 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.816974 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.818197 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.818564 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.818689 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.822317 (+ 3628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46240: BlockingRecv error: recv got EOF from 127.0.0.1:46240 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 02:47:19 dist-test-slave-n326 krb5kdc[2707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416439, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
I20260811 02:47:19.832508 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.823996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46242 (local address 127.21.236.190:37597)
0811 02:47:19.824135 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:19.824139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:19.824284 (+ 145us) server_negotiation.cc:407] Connection header received
0811 02:47:19.824427 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:19.824431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:19.824475 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:19.824569 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:19.825178 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.825699 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.826488 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.826812 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.826922 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.829483 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:19.829487 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:19.829490 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:19.829515 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:19.830778 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.831270 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.831275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.831277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.831366 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:19.831642 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:19.831646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:19.831648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:19.831828 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:19.831971 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:19.832226 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 02:47:19.832337 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:47:19.835553 2784 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:47:19.838581 2784 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:47:19.853192 3697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.847534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:39753 (local address 127.0.0.1:44554)
0811 02:47:19.847971 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:19.847984 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:19.848099 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:19.848638 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:19.848643 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:19.848664 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:19.848767 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.848771 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.850139 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.850143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.851565 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.851574 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.852618 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.852622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.852726 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.852766 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 02:47:19.852876 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 02:47:19.852987 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":105,"thread_start_us":103,"threads_started":1}
I20260811 02:47:19.853300 3701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.847017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.21.236.130:43023)
0811 02:47:19.848465 (+ 1448us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:19.848476 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:19.848499 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:47:19.848576 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:19.848581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:19.848751 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:19.848911 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:19.849106 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.850029 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.851753 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.852491 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.852640 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.852758 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:19.853078 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 02:47:19.853172 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1284,"thread_start_us":138,"threads_started":1}
I20260811 02:47:19.853243 3704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.846736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:43023 (local address 127.0.0.1:41220)
0811 02:47:19.847783 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:19.847795 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:19.847870 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:19.848890 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:19.848893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:19.848908 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:19.848985 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.848989 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.850162 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.850167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.851645 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.851655 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.852840 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.852844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.852931 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.852949 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 02:47:19.853040 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 02:47:19.853108 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"mutex_wait_us":448,"thread_start_us":98,"threads_started":1}
I20260811 02:47:19.853760 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.846736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:40905 (local address 127.0.0.1:41190)
0811 02:47:19.847611 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:19.847663 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:19.847790 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:19.849605 (+ 1815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:19.849608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:19.849627 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:19.849710 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.849715 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.851307 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.851311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.852181 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:19.852189 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.853364 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.853368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:19.853471 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.853493 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:47:19.853579 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 02:47:19.853642 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":82,"threads_started":1}
I20260811 02:47:19.853992 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.846938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41190 (local address 127.21.236.131:40905)
0811 02:47:19.849192 (+ 2254us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:19.849203 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:19.849224 (+ 21us) server_negotiation.cc:407] Connection header received
0811 02:47:19.849291 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:19.849296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:19.849457 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:19.849601 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:19.849826 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.851188 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.852298 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.853011 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.853177 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.853307 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:19.853618 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 02:47:19.853729 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2154,"thread_start_us":157,"threads_started":1}
I20260811 02:47:19.854076 3705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:19.847693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.21.236.129:39753)
0811 02:47:19.848046 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:19.848054 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:19.848079 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:47:19.848243 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:19.848252 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:19.848478 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:19.848624 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:19.848880 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.850003 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.851686 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:19.852491 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:19.852624 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:19.852758 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:19.853491 (+ 733us) server_negotiation.cc:299] Negotiation successful
0811 02:47:19.853602 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":59,"threads_started":1}
I20260811 02:47:19.856401 3298 tablet_service.cc:1467] Processing CreateTablet for tablet 2db3ea8837cf473d9d17d32348dbd26e (DEFAULT_TABLE table=test-table [id=b60c50b103ed4990bbac7eff164cb08d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:19.856366 3548 tablet_service.cc:1467] Processing CreateTablet for tablet 2db3ea8837cf473d9d17d32348dbd26e (DEFAULT_TABLE table=test-table [id=b60c50b103ed4990bbac7eff164cb08d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:19.857344 3298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2db3ea8837cf473d9d17d32348dbd26e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:19.857402 3548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2db3ea8837cf473d9d17d32348dbd26e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:19.858595 3038 tablet_service.cc:1467] Processing CreateTablet for tablet 2db3ea8837cf473d9d17d32348dbd26e (DEFAULT_TABLE table=test-table [id=b60c50b103ed4990bbac7eff164cb08d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:19.859587 3038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2db3ea8837cf473d9d17d32348dbd26e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:19.864861 3706 tablet_bootstrap.cc:492] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1: Bootstrap starting.
I20260811 02:47:19.865389 3711 tablet_bootstrap.cc:492] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f: Bootstrap starting.
I20260811 02:47:19.866246 3712 tablet_bootstrap.cc:492] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025: Bootstrap starting.
I20260811 02:47:19.867486 3711 tablet_bootstrap.cc:654] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:19.867507 3706 tablet_bootstrap.cc:654] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:19.868155 3712 tablet_bootstrap.cc:654] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:19.868412 3711 log.cc:824] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:19.868412 3706 log.cc:824] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:19.868942 3712 log.cc:824] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:19.870584 3706 tablet_bootstrap.cc:492] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1: No bootstrap required, opened a new log
I20260811 02:47:19.870795 3706 ts_tablet_manager.cc:1397] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 02:47:19.870800 3712 tablet_bootstrap.cc:492] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025: No bootstrap required, opened a new log
I20260811 02:47:19.870800 3711 tablet_bootstrap.cc:492] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f: No bootstrap required, opened a new log
I20260811 02:47:19.870977 3711 ts_tablet_manager.cc:1397] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:47:19.870977 3712 ts_tablet_manager.cc:1397] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 02:47:19.873742 3706 raft_consensus.cc:348] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } }
I20260811 02:47:19.873986 3706 raft_consensus.cc:374] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:19.874040 3706 raft_consensus.cc:729] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f5fd4a5c33444da5694e653491cdd1, State: Initialized, Role: FOLLOWER
I20260811 02:47:19.874465 3706 consensus_queue.cc:260] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } }
I20260811 02:47:19.874934 3711 raft_consensus.cc:348] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } }
I20260811 02:47:19.875146 3419 heartbeater.cc:503] Master 127.21.236.190:37597 was elected leader, sending a full tablet report...
I20260811 02:47:19.875195 3711 raft_consensus.cc:374] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:19.875247 3711 raft_consensus.cc:729] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6c99522e0164493ab63bd6137c8e85f, State: Initialized, Role: FOLLOWER
I20260811 02:47:19.875504 3706 ts_tablet_manager.cc:1428] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:47:19.875500 3712 raft_consensus.cc:348] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } }
I20260811 02:47:19.875723 3712 raft_consensus.cc:374] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:19.875663 3711 consensus_queue.cc:260] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } }
I20260811 02:47:19.875766 3712 raft_consensus.cc:729] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65c1aaacde848ebaf5ce2df1b01e025, State: Initialized, Role: FOLLOWER
I20260811 02:47:19.876196 3712 consensus_queue.cc:260] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } }
I20260811 02:47:19.876634 3711 ts_tablet_manager.cc:1428] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:47:19.876959 3660 heartbeater.cc:503] Master 127.21.236.190:37597 was elected leader, sending a full tablet report...
I20260811 02:47:19.877245 3161 heartbeater.cc:503] Master 127.21.236.190:37597 was elected leader, sending a full tablet report...
I20260811 02:47:19.878043 3712 ts_tablet_manager.cc:1428] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260811 02:47:20.035452 3662 tablet.cc:2369] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:47:20.054304 3420 tablet.cc:2369] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:47:20.069965 3163 tablet.cc:2369] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:47:20.139766 3720 raft_consensus.cc:482] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:47:20.139989 3720 raft_consensus.cc:504] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } }
I20260811 02:47:20.141214 3720 leader_election.cc:288] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6c99522e0164493ab63bd6137c8e85f (127.21.236.131:40905), 75f5fd4a5c33444da5694e653491cdd1 (127.21.236.130:43023)
I20260811 02:47:20.145143 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.141671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:58805 (local address 127.21.236.131:40905)
0811 02:47:20.141811 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:20.141816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:20.141991 (+ 175us) server_negotiation.cc:407] Connection header received
0811 02:47:20.142151 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:20.142156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:20.142204 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:20.142292 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:20.142574 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.143116 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.144191 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.144588 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.144719 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.144774 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:20.144951 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 02:47:20.145030 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:47:20.145246 3701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.141649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:49967 (local address 127.21.236.130:43023)
0811 02:47:20.141922 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:20.141927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:20.142051 (+ 124us) server_negotiation.cc:407] Connection header received
0811 02:47:20.142221 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:20.142226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:20.142272 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:20.142360 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:20.142652 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.143240 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.144259 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.144641 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.144768 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.144824 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:20.145041 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 02:47:20.145110 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260811 02:47:20.145336 3723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.141533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:43023 (local address 127.21.236.129:49967)
0811 02:47:20.141933 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:20.141948 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:20.142093 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:20.142380 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:20.142383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:20.142399 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:20.142532 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:20.142537 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.143350 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.143355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:20.144128 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:20.144137 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.144769 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.144773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:20.144902 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.144923 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 02:47:20.145009 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 02:47:20.145079 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":35,"thread_start_us":107,"threads_started":1}
I20260811 02:47:20.145159 3722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.141533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:40905 (local address 127.21.236.129:58805)
0811 02:47:20.141894 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:20.141906 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:20.142039 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:20.142305 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:20.142308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:20.142323 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:20.142454 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:20.142460 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.143229 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.143234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:20.144058 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:20.144067 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.144712 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.144716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:20.144812 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.144835 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:47:20.144934 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 02:47:20.145006 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":140,"threads_started":1}
I20260811 02:47:20.145978 3337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2db3ea8837cf473d9d17d32348dbd26e" candidate_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f5fd4a5c33444da5694e653491cdd1" is_pre_election: true
I20260811 02:47:20.146219 3575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2db3ea8837cf473d9d17d32348dbd26e" candidate_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6c99522e0164493ab63bd6137c8e85f" is_pre_election: true
I20260811 02:47:20.146288 3337 raft_consensus.cc:2454] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65c1aaacde848ebaf5ce2df1b01e025 in term 0.
I20260811 02:47:20.146495 3575 raft_consensus.cc:2454] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65c1aaacde848ebaf5ce2df1b01e025 in term 0.
I20260811 02:47:20.146987 2953 leader_election.cc:302] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 75f5fd4a5c33444da5694e653491cdd1, e65c1aaacde848ebaf5ce2df1b01e025, e6c99522e0164493ab63bd6137c8e85f; no voters:
I20260811 02:47:20.147305 3720 raft_consensus.cc:2781] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:47:20.147392 3720 raft_consensus.cc:482] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:47:20.147437 3720 raft_consensus.cc:3037] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:20.148483 3720 raft_consensus.cc:504] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } }
I20260811 02:47:20.148869 3720 leader_election.cc:288] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [CANDIDATE]: Term 1 election: Requested vote from peers e6c99522e0164493ab63bd6137c8e85f (127.21.236.131:40905), 75f5fd4a5c33444da5694e653491cdd1 (127.21.236.130:43023)
I20260811 02:47:20.149257 3575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2db3ea8837cf473d9d17d32348dbd26e" candidate_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6c99522e0164493ab63bd6137c8e85f"
I20260811 02:47:20.149389 3575 raft_consensus.cc:3037] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:20.149443 3337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2db3ea8837cf473d9d17d32348dbd26e" candidate_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f5fd4a5c33444da5694e653491cdd1"
I20260811 02:47:20.149577 3337 raft_consensus.cc:3037] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:20.150584 3337 raft_consensus.cc:2454] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65c1aaacde848ebaf5ce2df1b01e025 in term 1.
I20260811 02:47:20.150857 3575 raft_consensus.cc:2454] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65c1aaacde848ebaf5ce2df1b01e025 in term 1.
I20260811 02:47:20.150946 2952 leader_election.cc:302] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f5fd4a5c33444da5694e653491cdd1, e65c1aaacde848ebaf5ce2df1b01e025; no voters:
I20260811 02:47:20.151124 3720 raft_consensus.cc:2781] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:20.151365 3720 raft_consensus.cc:686] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 LEADER]: Becoming Leader. State: Replica: e65c1aaacde848ebaf5ce2df1b01e025, State: Running, Role: LEADER
I20260811 02:47:20.151691 3720 consensus_queue.cc:237] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } }
I20260811 02:47:20.154875 2780 catalog_manager.cc:5495] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 reported cstate change: term changed from 0 to 1, leader changed from <none> to e65c1aaacde848ebaf5ce2df1b01e025 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } health_report { overall_health: HEALTHY } } }
I20260811 02:47:20.217231 3705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.213657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.21.236.129:39753)
0811 02:47:20.213784 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:20.213788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:20.213802 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:47:20.213898 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:20.213900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:20.213955 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:20.214052 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:47:20.214323 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.214844 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.215800 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.216010 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.216090 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.216373 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:47:20.216834 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:47:20.216948 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:20.217068 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 02:47:20.217120 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 02:47:20.224054 3575 raft_consensus.cc:1261] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Refusing update from remote peer e65c1aaacde848ebaf5ce2df1b01e025: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:20.224056 3337 raft_consensus.cc:1261] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Refusing update from remote peer e65c1aaacde848ebaf5ce2df1b01e025: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:20.224807 3720 consensus_queue.cc:1035] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:20.224996 3724 consensus_queue.cc:1035] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:20.232520 3733 mvcc.cc:205] Tried to move back new op lower bound from 7317161739148845056 to 7317161738863828992. Current Snapshot: MvccSnapshot[applied={T|T < 7317161739148845056}]
I20260811 02:47:20.234339 3735 mvcc.cc:205] Tried to move back new op lower bound from 7317161739148845056 to 7317161738863828992. Current Snapshot: MvccSnapshot[applied={T|T < 7317161739148845056}]
I20260811 02:47:20.234514 3734 mvcc.cc:205] Tried to move back new op lower bound from 7317161739148845056 to 7317161738863828992. Current Snapshot: MvccSnapshot[applied={T|T < 7317161739148845056}]
I20260811 02:47:20.241618 2780 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:20.241768 2780 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:20.244310 2780 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3: Sending DeleteTablet for 3 replicas of tablet 2db3ea8837cf473d9d17d32348dbd26e
I20260811 02:47:20.245055 3548 tablet_service.cc:1514] Processing DeleteTablet for tablet 2db3ea8837cf473d9d17d32348dbd26e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:20 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:41190
I20260811 02:47:20.245059 3298 tablet_service.cc:1514] Processing DeleteTablet for tablet 2db3ea8837cf473d9d17d32348dbd26e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:20 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:41220
I20260811 02:47:20.245440 3038 tablet_service.cc:1514] Processing DeleteTablet for tablet 2db3ea8837cf473d9d17d32348dbd26e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:20 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:44554
I20260811 02:47:20.245916 3753 tablet_replica.cc:330] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025: stopping tablet replica
I20260811 02:47:20.245916 3752 tablet_replica.cc:330] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f: stopping tablet replica
I20260811 02:47:20.246138 2780 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 02:47:20.246263 3752 raft_consensus.cc:2229] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:20.246225 3753 raft_consensus.cc:2229] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 LEADER]: Raft consensus shutting down.
W20260811 02:47:20.246541 2780 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:47:20.246564 3752 raft_consensus.cc:2258] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:20.246923 3753 raft_consensus.cc:2258] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:20.248304 3753 ts_tablet_manager.cc:1905] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:20.250886 3752 ts_tablet_manager.cc:1905] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:20.251461 3548 tablet_service.cc:1467] Processing CreateTablet for tablet 88e186ef92bc4f32816ee0a6b21dd2d1 (DEFAULT_TABLE table=test-table [id=7ac67e1708eb4bb0b0a9a967f509f3d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:20.251459 3298 tablet_service.cc:1467] Processing CreateTablet for tablet 88e186ef92bc4f32816ee0a6b21dd2d1 (DEFAULT_TABLE table=test-table [id=7ac67e1708eb4bb0b0a9a967f509f3d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:20.251729 3548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e186ef92bc4f32816ee0a6b21dd2d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:20.251729 3298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e186ef92bc4f32816ee0a6b21dd2d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:20.251688 3038 tablet_service.cc:1467] Processing CreateTablet for tablet 88e186ef92bc4f32816ee0a6b21dd2d1 (DEFAULT_TABLE table=test-table [id=7ac67e1708eb4bb0b0a9a967f509f3d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:20.251961 3038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88e186ef92bc4f32816ee0a6b21dd2d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:20.253655 3712 tablet_bootstrap.cc:492] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025: Bootstrap starting.
I20260811 02:47:20.253986 3711 tablet_bootstrap.cc:492] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f: Bootstrap starting.
I20260811 02:47:20.254048 3706 tablet_bootstrap.cc:492] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1: Bootstrap starting.
I20260811 02:47:20.254673 3712 tablet_bootstrap.cc:654] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:20.254977 3711 tablet_bootstrap.cc:654] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:20.255028 3706 tablet_bootstrap.cc:654] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:20.256175 3711 tablet_bootstrap.cc:492] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f: No bootstrap required, opened a new log
I20260811 02:47:20.256166 3753 ts_tablet_manager.cc:1918] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:20.256280 3753 log.cc:1196] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/wal/wals/2db3ea8837cf473d9d17d32348dbd26e
I20260811 02:47:20.256286 3711 ts_tablet_manager.cc:1397] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 02:47:20.256699 3753 ts_tablet_manager.cc:1939] T 2db3ea8837cf473d9d17d32348dbd26e P e65c1aaacde848ebaf5ce2df1b01e025: Deleting consensus metadata
I20260811 02:47:20.256795 3711 raft_consensus.cc:348] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.256909 3711 raft_consensus.cc:374] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:20.256947 3711 raft_consensus.cc:729] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6c99522e0164493ab63bd6137c8e85f, State: Initialized, Role: FOLLOWER
I20260811 02:47:20.257089 3711 consensus_queue.cc:260] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.257380 3711 ts_tablet_manager.cc:1428] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 02:47:20.258050 3752 ts_tablet_manager.cc:1918] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:20.258055 2763 catalog_manager.cc:4841] TS e65c1aaacde848ebaf5ce2df1b01e025 (127.21.236.129:39753): tablet 2db3ea8837cf473d9d17d32348dbd26e (table test-table [id=b60c50b103ed4990bbac7eff164cb08d]) successfully deleted
I20260811 02:47:20.258150 3752 log.cc:1196] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/2db3ea8837cf473d9d17d32348dbd26e
I20260811 02:47:20.258458 3752 ts_tablet_manager.cc:1939] T 2db3ea8837cf473d9d17d32348dbd26e P e6c99522e0164493ab63bd6137c8e85f: Deleting consensus metadata
I20260811 02:47:20.258606 3706 tablet_bootstrap.cc:492] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1: No bootstrap required, opened a new log
I20260811 02:47:20.258700 3706 ts_tablet_manager.cc:1397] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 02:47:20.259151 3706 raft_consensus.cc:348] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.259265 3706 raft_consensus.cc:374] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:20.259308 3706 raft_consensus.cc:729] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f5fd4a5c33444da5694e653491cdd1, State: Initialized, Role: FOLLOWER
I20260811 02:47:20.259435 3706 consensus_queue.cc:260] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.259745 3706 ts_tablet_manager.cc:1428] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:47:20.260726 3712 tablet_bootstrap.cc:492] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025: No bootstrap required, opened a new log
I20260811 02:47:20.260828 3712 ts_tablet_manager.cc:1397] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 02:47:20.261309 3712 raft_consensus.cc:348] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.261454 3712 raft_consensus.cc:374] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:20.261489 3712 raft_consensus.cc:729] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65c1aaacde848ebaf5ce2df1b01e025, State: Initialized, Role: FOLLOWER
I20260811 02:47:20.261631 3712 consensus_queue.cc:260] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.261915 3712 ts_tablet_manager.cc:1428] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:47:20.262547 3750 tablet_replica.cc:330] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1: stopping tablet replica
I20260811 02:47:20.262863 3750 raft_consensus.cc:2229] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:20.264237 3750 raft_consensus.cc:2258] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:20.264213 2761 catalog_manager.cc:4841] TS e6c99522e0164493ab63bd6137c8e85f (127.21.236.131:40905): tablet 2db3ea8837cf473d9d17d32348dbd26e (table test-table [id=b60c50b103ed4990bbac7eff164cb08d]) successfully deleted
I20260811 02:47:20.265563 3750 ts_tablet_manager.cc:1905] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:20.268255 3750 ts_tablet_manager.cc:1918] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:20.268335 3750 log.cc:1196] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/2db3ea8837cf473d9d17d32348dbd26e
I20260811 02:47:20.268646 3750 ts_tablet_manager.cc:1939] T 2db3ea8837cf473d9d17d32348dbd26e P 75f5fd4a5c33444da5694e653491cdd1: Deleting consensus metadata
I20260811 02:47:20.269677 2756 catalog_manager.cc:4841] TS 75f5fd4a5c33444da5694e653491cdd1 (127.21.236.130:43023): tablet 2db3ea8837cf473d9d17d32348dbd26e (table test-table [id=b60c50b103ed4990bbac7eff164cb08d]) successfully deleted
I20260811 02:47:20.322027 3717 raft_consensus.cc:482] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:47:20.322201 3717 raft_consensus.cc:504] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.323550 3717 leader_election.cc:288] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65c1aaacde848ebaf5ce2df1b01e025 (127.21.236.129:39753), 75f5fd4a5c33444da5694e653491cdd1 (127.21.236.130:43023)
I20260811 02:47:20.329632 3701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.324258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:34477 (local address 127.21.236.130:43023)
0811 02:47:20.324380 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:20.324385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:20.324509 (+ 124us) server_negotiation.cc:407] Connection header received
0811 02:47:20.324773 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:20.324779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:20.324830 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:20.324882 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:20.325446 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.326231 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.328165 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.328784 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.328885 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.328939 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:20.329423 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 02:47:20.329499 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:47:20.329632 3705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.324014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:44215 (local address 127.21.236.129:39753)
0811 02:47:20.324146 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:20.324150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:20.324640 (+ 490us) server_negotiation.cc:407] Connection header received
0811 02:47:20.324805 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:20.324808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:20.324855 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:20.324903 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:20.325421 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.326210 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.328248 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.328841 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.328915 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.328961 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:20.329423 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 02:47:20.329502 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:47:20.330040 3760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.323833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:43023 (local address 127.21.236.131:34477)
0811 02:47:20.324393 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:20.324412 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:20.324632 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:20.325036 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:20.325039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:20.325068 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:20.325320 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:20.325331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.326375 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.326379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:20.328004 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:20.328027 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.329062 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.329065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:20.329267 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.329311 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 02:47:20.329780 (+ 469us) client_negotiation.cc:240] Negotiation successful
0811 02:47:20.329889 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":242,"spinlock_wait_cycles":5376,"thread_start_us":75,"threads_started":1}
I20260811 02:47:20.330451 3758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.323833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:39753 (local address 127.21.236.131:44215)
0811 02:47:20.324565 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:20.324578 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:20.324697 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:20.325036 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:20.325039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:20.325073 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:20.325280 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:20.325291 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.326392 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.326394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:20.328124 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:20.328136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.329061 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.329065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:20.329244 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.329289 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 02:47:20.330041 (+ 752us) client_negotiation.cc:240] Negotiation successful
0811 02:47:20.330305 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":586,"spinlock_wait_cycles":16128,"thread_start_us":108,"threads_started":1}
I20260811 02:47:20.330554 3337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e186ef92bc4f32816ee0a6b21dd2d1" candidate_uuid: "e6c99522e0164493ab63bd6137c8e85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f5fd4a5c33444da5694e653491cdd1" is_pre_election: true
I20260811 02:47:20.330714 3337 raft_consensus.cc:2454] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6c99522e0164493ab63bd6137c8e85f in term 0.
I20260811 02:47:20.331211 3079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e186ef92bc4f32816ee0a6b21dd2d1" candidate_uuid: "e6c99522e0164493ab63bd6137c8e85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" is_pre_election: true
I20260811 02:47:20.331231 3465 leader_election.cc:302] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f5fd4a5c33444da5694e653491cdd1, e6c99522e0164493ab63bd6137c8e85f; no voters:
I20260811 02:47:20.331440 3079 raft_consensus.cc:2454] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6c99522e0164493ab63bd6137c8e85f in term 0.
I20260811 02:47:20.331493 3717 raft_consensus.cc:2781] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:47:20.331549 3717 raft_consensus.cc:482] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:47:20.331588 3717 raft_consensus.cc:3037] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:20.332888 3717 raft_consensus.cc:504] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.333251 3717 leader_election.cc:288] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [CANDIDATE]: Term 1 election: Requested vote from peers e65c1aaacde848ebaf5ce2df1b01e025 (127.21.236.129:39753), 75f5fd4a5c33444da5694e653491cdd1 (127.21.236.130:43023)
I20260811 02:47:20.333940 3079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e186ef92bc4f32816ee0a6b21dd2d1" candidate_uuid: "e6c99522e0164493ab63bd6137c8e85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65c1aaacde848ebaf5ce2df1b01e025"
I20260811 02:47:20.334071 3079 raft_consensus.cc:3037] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:20.334048 3337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88e186ef92bc4f32816ee0a6b21dd2d1" candidate_uuid: "e6c99522e0164493ab63bd6137c8e85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f5fd4a5c33444da5694e653491cdd1"
I20260811 02:47:20.334158 3337 raft_consensus.cc:3037] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:20.335250 3079 raft_consensus.cc:2454] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6c99522e0164493ab63bd6137c8e85f in term 1.
I20260811 02:47:20.335250 3337 raft_consensus.cc:2454] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6c99522e0164493ab63bd6137c8e85f in term 1.
I20260811 02:47:20.335570 3467 leader_election.cc:302] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e65c1aaacde848ebaf5ce2df1b01e025, e6c99522e0164493ab63bd6137c8e85f; no voters:
I20260811 02:47:20.335754 3717 raft_consensus.cc:2781] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:20.335951 3717 raft_consensus.cc:686] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [term 1 LEADER]: Becoming Leader. State: Replica: e6c99522e0164493ab63bd6137c8e85f, State: Running, Role: LEADER
I20260811 02:47:20.336205 3717 consensus_queue.cc:237] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.338497 2780 catalog_manager.cc:5495] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f reported cstate change: term changed from 0 to 1, leader changed from <none> to e6c99522e0164493ab63bd6137c8e85f (127.21.236.131). New cstate: current_term: 1 leader_uuid: "e6c99522e0164493ab63bd6137c8e85f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } health_report { overall_health: HEALTHY } } }
Aug 11 02:47:20 dist-test-slave-n326 krb5kdc[2707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416438, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.190@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
I20260811 02:47:20.395573 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.385308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:37597 (local address 127.0.0.1:46258)
0811 02:47:20.385613 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:20.385633 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:20.385740 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:20.386067 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:20.386070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:20.386449 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:20.386550 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:20.386556 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.387786 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.387789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:20.388425 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:20.388434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.389124 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.389127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:20.389191 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.389848 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:20.389859 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:20.391718 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:20.393820 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:20.393829 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:20.393834 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:20.394067 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:20.394411 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:20.394413 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:20.394415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:20.394495 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:20.395022 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:20.395030 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:20.395199 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 02:47:20.395293 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 02:47:20.395404 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":129,"threads_started":1}
I20260811 02:47:20.395798 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.385381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46258 (local address 127.21.236.190:37597)
0811 02:47:20.385696 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:20.385706 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:20.385728 (+ 22us) server_negotiation.cc:407] Connection header received
0811 02:47:20.385829 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:20.385833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:20.385889 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:20.386019 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:20.386701 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.387672 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.388559 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.388958 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.389114 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.391878 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:20.391882 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:20.391886 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:20.391914 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:20.393688 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:20.394182 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:20.394186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:20.394188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:20.394304 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:20.394630 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:20.394634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:20.394636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:20.394899 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:20.395021 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:20.395404 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 02:47:20.395561 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":125,"threads_started":1}
I20260811 02:47:20.400183 2780 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:46258:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 02:47:20.400951 2780 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:47:20.406095 3038 tablet_service.cc:1467] Processing CreateTablet for tablet 18f382db505e451c98b2312adf8449a2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=53e8d4db4a1e41c094ec3442a4572a05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:47:20.406122 3548 tablet_service.cc:1467] Processing CreateTablet for tablet 18f382db505e451c98b2312adf8449a2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=53e8d4db4a1e41c094ec3442a4572a05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:47:20.406558 3038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18f382db505e451c98b2312adf8449a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:20.406571 3548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18f382db505e451c98b2312adf8449a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:20.406443 3298 tablet_service.cc:1467] Processing CreateTablet for tablet 18f382db505e451c98b2312adf8449a2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=53e8d4db4a1e41c094ec3442a4572a05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 02:47:20.406804 3298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18f382db505e451c98b2312adf8449a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:20.409797 3711 tablet_bootstrap.cc:492] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f: Bootstrap starting.
I20260811 02:47:20.410149 3706 tablet_bootstrap.cc:492] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1: Bootstrap starting.
I20260811 02:47:20.410867 3711 tablet_bootstrap.cc:654] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:20.411202 3706 tablet_bootstrap.cc:654] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:20.411657 3712 tablet_bootstrap.cc:492] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025: Bootstrap starting.
I20260811 02:47:20.412581 3712 tablet_bootstrap.cc:654] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:20.412773 3711 tablet_bootstrap.cc:492] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f: No bootstrap required, opened a new log
I20260811 02:47:20.412859 3711 ts_tablet_manager.cc:1397] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 02:47:20.413335 3711 raft_consensus.cc:348] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } }
I20260811 02:47:20.413482 3711 raft_consensus.cc:374] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:20.413519 3711 raft_consensus.cc:729] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6c99522e0164493ab63bd6137c8e85f, State: Initialized, Role: FOLLOWER
I20260811 02:47:20.413688 3711 consensus_queue.cc:260] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } }
I20260811 02:47:20.413781 3712 tablet_bootstrap.cc:492] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025: No bootstrap required, opened a new log
I20260811 02:47:20.413873 3712 ts_tablet_manager.cc:1397] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:47:20.413937 3717 tablet_replica.cc:439] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } }
I20260811 02:47:20.414083 3717 tablet_replica.cc:442] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:47:20.414736 3706 tablet_bootstrap.cc:492] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1: No bootstrap required, opened a new log
I20260811 02:47:20.414678 3712 raft_consensus.cc:348] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } }
I20260811 02:47:20.414794 3712 raft_consensus.cc:374] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:20.414824 3706 ts_tablet_manager.cc:1397] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260811 02:47:20.414827 3712 raft_consensus.cc:729] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65c1aaacde848ebaf5ce2df1b01e025, State: Initialized, Role: FOLLOWER
I20260811 02:47:20.415166 3712 consensus_queue.cc:260] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } }
I20260811 02:47:20.415320 3706 raft_consensus.cc:348] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } }
I20260811 02:47:20.415422 3706 raft_consensus.cc:374] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:20.415454 3706 raft_consensus.cc:729] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f5fd4a5c33444da5694e653491cdd1, State: Initialized, Role: FOLLOWER
I20260811 02:47:20.415581 3706 consensus_queue.cc:260] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } }
I20260811 02:47:20.415679 3743 tablet_replica.cc:439] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } }
I20260811 02:47:20.415810 3743 tablet_replica.cc:442] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:47:20.415853 3706 ts_tablet_manager.cc:1428] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 02:47:20.416114 3716 tablet_replica.cc:439] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } }
I20260811 02:47:20.416252 3716 tablet_replica.cc:442] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:47:20.416441 3711 ts_tablet_manager.cc:1428] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260811 02:47:20.417582 3712 ts_tablet_manager.cc:1428] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260811 02:47:20.595546 3717 raft_consensus.cc:482] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:47:20.595714 3717 raft_consensus.cc:504] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } }
I20260811 02:47:20.596278 3717 leader_election.cc:288] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65c1aaacde848ebaf5ce2df1b01e025 (127.21.236.129:39753), 75f5fd4a5c33444da5694e653491cdd1 (127.21.236.130:43023)
I20260811 02:47:20.596923 3337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18f382db505e451c98b2312adf8449a2" candidate_uuid: "e6c99522e0164493ab63bd6137c8e85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f5fd4a5c33444da5694e653491cdd1" is_pre_election: true
I20260811 02:47:20.596968 3079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18f382db505e451c98b2312adf8449a2" candidate_uuid: "e6c99522e0164493ab63bd6137c8e85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" is_pre_election: true
I20260811 02:47:20.597134 3337 raft_consensus.cc:2454] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6c99522e0164493ab63bd6137c8e85f in term 0.
I20260811 02:47:20.597144 3079 raft_consensus.cc:2454] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6c99522e0164493ab63bd6137c8e85f in term 0.
I20260811 02:47:20.597522 3465 leader_election.cc:302] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f5fd4a5c33444da5694e653491cdd1, e6c99522e0164493ab63bd6137c8e85f; no voters:
I20260811 02:47:20.597754 3717 raft_consensus.cc:2781] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:47:20.597831 3717 raft_consensus.cc:482] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:47:20.597923 3717 raft_consensus.cc:3037] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:20.599021 3717 raft_consensus.cc:504] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } }
I20260811 02:47:20.599445 3717 leader_election.cc:288] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [CANDIDATE]: Term 1 election: Requested vote from peers e65c1aaacde848ebaf5ce2df1b01e025 (127.21.236.129:39753), 75f5fd4a5c33444da5694e653491cdd1 (127.21.236.130:43023)
I20260811 02:47:20.599840 3079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18f382db505e451c98b2312adf8449a2" candidate_uuid: "e6c99522e0164493ab63bd6137c8e85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65c1aaacde848ebaf5ce2df1b01e025"
I20260811 02:47:20.599846 3337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18f382db505e451c98b2312adf8449a2" candidate_uuid: "e6c99522e0164493ab63bd6137c8e85f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f5fd4a5c33444da5694e653491cdd1"
I20260811 02:47:20.599987 3079 raft_consensus.cc:3037] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:20.600023 3337 raft_consensus.cc:3037] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:20.601277 3337 raft_consensus.cc:2454] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6c99522e0164493ab63bd6137c8e85f in term 1.
I20260811 02:47:20.601413 3079 raft_consensus.cc:2454] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6c99522e0164493ab63bd6137c8e85f in term 1.
I20260811 02:47:20.601608 3465 leader_election.cc:302] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f5fd4a5c33444da5694e653491cdd1, e6c99522e0164493ab63bd6137c8e85f; no voters:
I20260811 02:47:20.601800 3717 raft_consensus.cc:2781] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:20.601904 3717 raft_consensus.cc:686] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [term 1 LEADER]: Becoming Leader. State: Replica: e6c99522e0164493ab63bd6137c8e85f, State: Running, Role: LEADER
I20260811 02:47:20.602063 3717 consensus_queue.cc:237] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } }
I20260811 02:47:20.602803 3761 tablet_replica.cc:439] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f: TxnStatusTablet state changed. Reason: New leader e6c99522e0164493ab63bd6137c8e85f. Latest consensus state: current_term: 1 leader_uuid: "e6c99522e0164493ab63bd6137c8e85f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } }
I20260811 02:47:20.602937 3761 tablet_replica.cc:442] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:47:20.603287 3785 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 02:47:20.603508 3785 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 02:47:20.603961 2783 catalog_manager.cc:5495] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f reported cstate change: term changed from 0 to 1, leader changed from <none> to e6c99522e0164493ab63bd6137c8e85f (127.21.236.131). New cstate: current_term: 1 leader_uuid: "e6c99522e0164493ab63bd6137c8e85f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } health_report { overall_health: UNKNOWN } } }
I20260811 02:47:20.658591 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.653779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41192 (local address 127.21.236.131:40905)
0811 02:47:20.654071 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:20.654075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:20.654089 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:47:20.654159 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:20.654161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:20.654217 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:20.654314 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:47:20.654671 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.655364 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.656804 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.657064 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.657164 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.657565 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:47:20.658128 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:47:20.658228 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:20.658366 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 02:47:20.658444 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":124,"threads_started":1}
I20260811 02:47:20.658601 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.653270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:40905 (local address 127.0.0.1:41192)
0811 02:47:20.653523 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:20.653541 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:20.653690 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:20.654371 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:20.654375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:20.654386 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:47:20.654520 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:20.654527 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.655512 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.655516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:20.656676 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:20.656688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.657195 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.657199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:20.657309 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.657370 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:47:20.658244 (+ 874us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:47:20.658262 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 02:47:20.658344 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 02:47:20.658436 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 02:47:20.661834 3337 raft_consensus.cc:1261] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Refusing update from remote peer e6c99522e0164493ab63bd6137c8e85f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:20.662081 3079 raft_consensus.cc:1261] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Refusing update from remote peer e6c99522e0164493ab63bd6137c8e85f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:20.662421 3761 consensus_queue.cc:1035] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:20.662706 3717 consensus_queue.cc:1035] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:20.664381 3716 tablet_replica.cc:439] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1: TxnStatusTablet state changed. Reason: New leader e6c99522e0164493ab63bd6137c8e85f. Latest consensus state: current_term: 1 leader_uuid: "e6c99522e0164493ab63bd6137c8e85f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } }
I20260811 02:47:20.664527 3716 tablet_replica.cc:442] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:47:20.665606 3743 tablet_replica.cc:439] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025: TxnStatusTablet state changed. Reason: New leader e6c99522e0164493ab63bd6137c8e85f. Latest consensus state: current_term: 1 leader_uuid: "e6c99522e0164493ab63bd6137c8e85f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } }
I20260811 02:47:20.665751 3743 tablet_replica.cc:442] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:47:20.666252 3717 tablet_replica.cc:439] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e6c99522e0164493ab63bd6137c8e85f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } }
I20260811 02:47:20.666361 3717 tablet_replica.cc:442] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:47:20.667943 3716 tablet_replica.cc:439] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e6c99522e0164493ab63bd6137c8e85f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } }
I20260811 02:47:20.668058 3716 tablet_replica.cc:442] T 18f382db505e451c98b2312adf8449a2 P 75f5fd4a5c33444da5694e653491cdd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:47:20.670680 3761 tablet_replica.cc:439] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e6c99522e0164493ab63bd6137c8e85f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } }
I20260811 02:47:20.670812 3761 tablet_replica.cc:442] T 18f382db505e451c98b2312adf8449a2 P e6c99522e0164493ab63bd6137c8e85f: This TxnStatusTablet replica's current role is: LEADER
I20260811 02:47:20.671658 3743 tablet_replica.cc:439] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e6c99522e0164493ab63bd6137c8e85f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } }
I20260811 02:47:20.671801 3743 tablet_replica.cc:442] T 18f382db505e451c98b2312adf8449a2 P e65c1aaacde848ebaf5ce2df1b01e025: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 02:47:20.677435 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.673683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41194 (local address 127.21.236.131:40905)
0811 02:47:20.673891 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:20.673895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:20.673907 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:47:20.673948 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:20.673950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:20.673989 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:20.674057 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:47:20.674341 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.674936 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.676305 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.676543 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.676682 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.676970 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:47:20.677052 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:47:20.677111 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:20.677252 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 02:47:20.677325 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260811 02:47:20.689339 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.685253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:51329 (local address 127.21.236.131:40905)
0811 02:47:20.685380 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:20.685385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:20.685585 (+ 200us) server_negotiation.cc:407] Connection header received
0811 02:47:20.685712 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:20.685714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:20.685757 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:20.685823 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:47:20.686168 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.686689 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.688102 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.688313 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.688450 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.688863 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:47:20.688958 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:47:20.689021 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:20.689153 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 02:47:20.689206 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":50}
I20260811 02:47:20.689688 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.685183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:40905 (local address 127.21.236.131:51329)
0811 02:47:20.685493 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:20.685510 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:20.685620 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:20.685865 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:20.685869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:20.685878 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:47:20.686030 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:20.686036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.686808 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.686812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:20.687966 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:20.687978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.688509 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.688514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:20.688650 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.688711 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:47:20.689051 (+ 340us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:47:20.689058 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 02:47:20.689132 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 02:47:20.689219 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":3,"thread_start_us":111,"threads_started":1}
I20260811 02:47:20.696851 3337 raft_consensus.cc:1261] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Refusing update from remote peer e6c99522e0164493ab63bd6137c8e85f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:20.696914 3079 raft_consensus.cc:1261] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Refusing update from remote peer e6c99522e0164493ab63bd6137c8e85f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:20.697264 3795 consensus_queue.cc:1035] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:20.697422 3761 consensus_queue.cc:1035] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:20.738484 2780 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:20.738687 2780 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:20.740931 2780 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3: Sending DeleteTablet for 3 replicas of tablet 88e186ef92bc4f32816ee0a6b21dd2d1
I20260811 02:47:20.741570 3038 tablet_service.cc:1514] Processing DeleteTablet for tablet 88e186ef92bc4f32816ee0a6b21dd2d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:20 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:44554
I20260811 02:47:20.741659 3298 tablet_service.cc:1514] Processing DeleteTablet for tablet 88e186ef92bc4f32816ee0a6b21dd2d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:20 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:41220
I20260811 02:47:20.741729 3547 tablet_service.cc:1514] Processing DeleteTablet for tablet 88e186ef92bc4f32816ee0a6b21dd2d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:20 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:41190
I20260811 02:47:20.741778 3753 tablet_replica.cc:330] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025: stopping tablet replica
I20260811 02:47:20.741847 3750 tablet_replica.cc:330] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1: stopping tablet replica
I20260811 02:47:20.741884 3752 tablet_replica.cc:330] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f: stopping tablet replica
I20260811 02:47:20.741900 3753 raft_consensus.cc:2229] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:20.741953 3750 raft_consensus.cc:2229] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:20.741978 3752 raft_consensus.cc:2229] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:47:20.742022 3753 raft_consensus.cc:2258] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:20.742069 3750 raft_consensus.cc:2258] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:20.742343 3752 raft_consensus.cc:2258] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:20.742873 3750 ts_tablet_manager.cc:1905] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:20.743160 3752 ts_tablet_manager.cc:1905] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:20.743250 3753 ts_tablet_manager.cc:1905] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:20.745867 3752 ts_tablet_manager.cc:1918] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:47:20.745867 3753 ts_tablet_manager.cc:1918] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:47:20.745940 3753 log.cc:1196] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/wal/wals/88e186ef92bc4f32816ee0a6b21dd2d1
I20260811 02:47:20.745940 3752 log.cc:1196] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/88e186ef92bc4f32816ee0a6b21dd2d1
I20260811 02:47:20.746143 3750 ts_tablet_manager.cc:1918] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:47:20.746215 3752 ts_tablet_manager.cc:1939] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e6c99522e0164493ab63bd6137c8e85f: Deleting consensus metadata
I20260811 02:47:20.746215 3753 ts_tablet_manager.cc:1939] T 88e186ef92bc4f32816ee0a6b21dd2d1 P e65c1aaacde848ebaf5ce2df1b01e025: Deleting consensus metadata
I20260811 02:47:20.746234 3750 log.cc:1196] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/88e186ef92bc4f32816ee0a6b21dd2d1
I20260811 02:47:20.746519 3750 ts_tablet_manager.cc:1939] T 88e186ef92bc4f32816ee0a6b21dd2d1 P 75f5fd4a5c33444da5694e653491cdd1: Deleting consensus metadata
I20260811 02:47:20.747308 2756 catalog_manager.cc:4841] TS 75f5fd4a5c33444da5694e653491cdd1 (127.21.236.130:43023): tablet 88e186ef92bc4f32816ee0a6b21dd2d1 (table test-table [id=7ac67e1708eb4bb0b0a9a967f509f3d2]) successfully deleted
I20260811 02:47:20.747685 2761 catalog_manager.cc:4841] TS e6c99522e0164493ab63bd6137c8e85f (127.21.236.131:40905): tablet 88e186ef92bc4f32816ee0a6b21dd2d1 (table test-table [id=7ac67e1708eb4bb0b0a9a967f509f3d2]) successfully deleted
I20260811 02:47:20.751091 2763 catalog_manager.cc:4841] TS e65c1aaacde848ebaf5ce2df1b01e025 (127.21.236.129:39753): tablet 88e186ef92bc4f32816ee0a6b21dd2d1 (table test-table [id=7ac67e1708eb4bb0b0a9a967f509f3d2]) successfully deleted
Aug 11 02:47:20 dist-test-slave-n326 krb5kdc[2707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 02:47:20 dist-test-slave-n326 krb5kdc[2707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:20 dist-test-slave-n326 krb5kdc[2707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 02:47:20.775662 3775 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:20.768694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46268 (local address 127.21.236.190:37597)
0811 02:47:20.768842 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:20.768847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:20.768987 (+ 140us) server_negotiation.cc:407] Connection header received
0811 02:47:20.769121 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:20.769125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:20.769178 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:20.769276 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:20.769979 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.770797 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.771747 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.772102 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.772213 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.775505 (+ 3292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46268: BlockingRecv error: recv got EOF from 127.0.0.1:46268 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 02:47:20 dist-test-slave-n326 krb5kdc[2707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM
I20260811 02:47:20.786675 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.777430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46276 (local address 127.21.236.190:37597)
0811 02:47:20.777607 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:20.777612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:20.777764 (+ 152us) server_negotiation.cc:407] Connection header received
0811 02:47:20.777946 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:20.777950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:20.778001 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:20.778093 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:20.778803 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.779409 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.780281 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.780490 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.780612 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.783304 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:20.783308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:20.783311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:20.783344 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:20.784991 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:20.785415 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:20.785418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:20.785420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:20.785507 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:20.785793 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:20.785797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:20.785799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:20.785988 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:20.786131 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:20.786370 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 02:47:20.786496 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:47:20.797385 2780 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:47:20.797875 2780 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:47:20.803145 3038 tablet_service.cc:1467] Processing CreateTablet for tablet d68f9125004f40dbaf9f32d3e127f540 (DEFAULT_TABLE table=test-table [id=1091f7f428614735beea6a655c181df5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:20.803565 3038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68f9125004f40dbaf9f32d3e127f540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:20.806058 3712 tablet_bootstrap.cc:492] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025: Bootstrap starting.
I20260811 02:47:20.806074 3298 tablet_service.cc:1467] Processing CreateTablet for tablet d68f9125004f40dbaf9f32d3e127f540 (DEFAULT_TABLE table=test-table [id=1091f7f428614735beea6a655c181df5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:20.806344 3298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68f9125004f40dbaf9f32d3e127f540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:20.806923 3712 tablet_bootstrap.cc:654] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:20.807780 3547 tablet_service.cc:1467] Processing CreateTablet for tablet d68f9125004f40dbaf9f32d3e127f540 (DEFAULT_TABLE table=test-table [id=1091f7f428614735beea6a655c181df5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:20.808122 3547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68f9125004f40dbaf9f32d3e127f540. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:20.808903 3706 tablet_bootstrap.cc:492] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1: Bootstrap starting.
I20260811 02:47:20.809842 3706 tablet_bootstrap.cc:654] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:20.810701 3711 tablet_bootstrap.cc:492] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f: Bootstrap starting.
I20260811 02:47:20.811630 3706 tablet_bootstrap.cc:492] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1: No bootstrap required, opened a new log
I20260811 02:47:20.811641 3711 tablet_bootstrap.cc:654] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:20.811722 3706 ts_tablet_manager.cc:1397] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 02:47:20.812157 3706 raft_consensus.cc:348] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.812281 3706 raft_consensus.cc:374] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:20.812319 3706 raft_consensus.cc:729] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f5fd4a5c33444da5694e653491cdd1, State: Initialized, Role: FOLLOWER
I20260811 02:47:20.812438 3706 consensus_queue.cc:260] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.812558 3712 tablet_bootstrap.cc:492] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025: No bootstrap required, opened a new log
I20260811 02:47:20.812652 3712 ts_tablet_manager.cc:1397] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260811 02:47:20.812688 3706 ts_tablet_manager.cc:1428] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:47:20.813100 3712 raft_consensus.cc:348] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.813252 3712 raft_consensus.cc:374] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:20.813295 3712 raft_consensus.cc:729] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65c1aaacde848ebaf5ce2df1b01e025, State: Initialized, Role: FOLLOWER
I20260811 02:47:20.813323 3711 tablet_bootstrap.cc:492] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f: No bootstrap required, opened a new log
I20260811 02:47:20.813410 3711 ts_tablet_manager.cc:1397] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 02:47:20.813436 3712 consensus_queue.cc:260] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.813685 3712 ts_tablet_manager.cc:1428] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:47:20.813781 3711 raft_consensus.cc:348] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.813894 3711 raft_consensus.cc:374] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:20.813934 3711 raft_consensus.cc:729] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6c99522e0164493ab63bd6137c8e85f, State: Initialized, Role: FOLLOWER
I20260811 02:47:20.814062 3711 consensus_queue.cc:260] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.814306 3711 ts_tablet_manager.cc:1428] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:47:20.877327 3743 raft_consensus.cc:482] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:47:20.877463 3743 raft_consensus.cc:504] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.877961 3743 leader_election.cc:288] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75f5fd4a5c33444da5694e653491cdd1 (127.21.236.130:43023), e6c99522e0164493ab63bd6137c8e85f (127.21.236.131:40905)
I20260811 02:47:20.878404 3575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68f9125004f40dbaf9f32d3e127f540" candidate_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6c99522e0164493ab63bd6137c8e85f" is_pre_election: true
I20260811 02:47:20.878435 3335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68f9125004f40dbaf9f32d3e127f540" candidate_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f5fd4a5c33444da5694e653491cdd1" is_pre_election: true
I20260811 02:47:20.878575 3575 raft_consensus.cc:2454] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65c1aaacde848ebaf5ce2df1b01e025 in term 0.
I20260811 02:47:20.878575 3335 raft_consensus.cc:2454] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65c1aaacde848ebaf5ce2df1b01e025 in term 0.
I20260811 02:47:20.878871 2953 leader_election.cc:302] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e65c1aaacde848ebaf5ce2df1b01e025, e6c99522e0164493ab63bd6137c8e85f; no voters:
I20260811 02:47:20.879112 3743 raft_consensus.cc:2781] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:47:20.879179 3743 raft_consensus.cc:482] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:47:20.879230 3743 raft_consensus.cc:3037] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:20.880277 3743 raft_consensus.cc:504] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.880643 3743 leader_election.cc:288] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [CANDIDATE]: Term 1 election: Requested vote from peers 75f5fd4a5c33444da5694e653491cdd1 (127.21.236.130:43023), e6c99522e0164493ab63bd6137c8e85f (127.21.236.131:40905)
I20260811 02:47:20.881029 3575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68f9125004f40dbaf9f32d3e127f540" candidate_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6c99522e0164493ab63bd6137c8e85f"
I20260811 02:47:20.881016 3335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68f9125004f40dbaf9f32d3e127f540" candidate_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75f5fd4a5c33444da5694e653491cdd1"
I20260811 02:47:20.881168 3335 raft_consensus.cc:3037] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:20.881173 3575 raft_consensus.cc:3037] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:20.882092 3575 raft_consensus.cc:2454] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65c1aaacde848ebaf5ce2df1b01e025 in term 1.
I20260811 02:47:20.882401 3335 raft_consensus.cc:2454] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65c1aaacde848ebaf5ce2df1b01e025 in term 1.
I20260811 02:47:20.882414 2953 leader_election.cc:302] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e65c1aaacde848ebaf5ce2df1b01e025, e6c99522e0164493ab63bd6137c8e85f; no voters:
I20260811 02:47:20.882743 3743 raft_consensus.cc:2781] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:20.882979 3743 raft_consensus.cc:686] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 LEADER]: Becoming Leader. State: Replica: e65c1aaacde848ebaf5ce2df1b01e025, State: Running, Role: LEADER
I20260811 02:47:20.883167 3743 consensus_queue.cc:237] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.884800 2783 catalog_manager.cc:5495] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 reported cstate change: term changed from 0 to 1, leader changed from <none> to e65c1aaacde848ebaf5ce2df1b01e025 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } health_report { overall_health: UNKNOWN } } }
I20260811 02:47:20.895097 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:20.891794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44572 (local address 127.21.236.129:39753)
0811 02:47:20.892063 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:20.892069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:20.892083 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:47:20.892124 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:20.892126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:20.892173 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:20.892261 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:47:20.892507 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.893095 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.893962 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:20.894202 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:20.894305 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:20.894563 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:47:20.894641 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:47:20.894721 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:20.894846 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 02:47:20.894902 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":92,"threads_started":1}
I20260811 02:47:20.896972 3575 raft_consensus.cc:1261] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Refusing update from remote peer e65c1aaacde848ebaf5ce2df1b01e025: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:20.896979 3335 raft_consensus.cc:1261] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Refusing update from remote peer e65c1aaacde848ebaf5ce2df1b01e025: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:20.897423 3826 consensus_queue.cc:1035] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:20.897585 3743 consensus_queue.cc:1035] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:20.906376 2780 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:20.906502 2780 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:20.908538 2780 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3: Sending DeleteTablet for 3 replicas of tablet d68f9125004f40dbaf9f32d3e127f540
I20260811 02:47:20.909169 3038 tablet_service.cc:1514] Processing DeleteTablet for tablet d68f9125004f40dbaf9f32d3e127f540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:20 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:44554
I20260811 02:47:20.909251 3547 tablet_service.cc:1514] Processing DeleteTablet for tablet d68f9125004f40dbaf9f32d3e127f540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:20 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:41190
I20260811 02:47:20.909358 3753 tablet_replica.cc:330] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025: stopping tablet replica
I20260811 02:47:20.909428 3752 tablet_replica.cc:330] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f: stopping tablet replica
I20260811 02:47:20.909514 3753 raft_consensus.cc:2229] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:47:20.909543 3752 raft_consensus.cc:2229] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:20.909669 3752 raft_consensus.cc:2258] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:20.909734 3753 raft_consensus.cc:2258] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:20.909724 3298 tablet_service.cc:1514] Processing DeleteTablet for tablet d68f9125004f40dbaf9f32d3e127f540 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:20 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:41220
I20260811 02:47:20.909965 3750 tablet_replica.cc:330] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1: stopping tablet replica
I20260811 02:47:20.910108 3750 raft_consensus.cc:2229] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:20.910251 3750 raft_consensus.cc:2258] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:20.910578 3753 ts_tablet_manager.cc:1905] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:20.910578 3752 ts_tablet_manager.cc:1905] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:20.911330 2780 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:47:20.911746 2780 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:47:20.913050 3752 ts_tablet_manager.cc:1918] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:20.913053 3753 ts_tablet_manager.cc:1918] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:20.913125 3753 log.cc:1196] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-0/wal/wals/d68f9125004f40dbaf9f32d3e127f540
I20260811 02:47:20.913125 3752 log.cc:1196] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/d68f9125004f40dbaf9f32d3e127f540
I20260811 02:47:20.913391 3753 ts_tablet_manager.cc:1939] T d68f9125004f40dbaf9f32d3e127f540 P e65c1aaacde848ebaf5ce2df1b01e025: Deleting consensus metadata
I20260811 02:47:20.913391 3752 ts_tablet_manager.cc:1939] T d68f9125004f40dbaf9f32d3e127f540 P e6c99522e0164493ab63bd6137c8e85f: Deleting consensus metadata
I20260811 02:47:20.914093 2763 catalog_manager.cc:4841] TS e65c1aaacde848ebaf5ce2df1b01e025 (127.21.236.129:39753): tablet d68f9125004f40dbaf9f32d3e127f540 (table test-table [id=1091f7f428614735beea6a655c181df5]) successfully deleted
I20260811 02:47:20.914412 2761 catalog_manager.cc:4841] TS e6c99522e0164493ab63bd6137c8e85f (127.21.236.131:40905): tablet d68f9125004f40dbaf9f32d3e127f540 (table test-table [id=1091f7f428614735beea6a655c181df5]) successfully deleted
I20260811 02:47:20.916184 3750 ts_tablet_manager.cc:1905] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:20.918757 3750 ts_tablet_manager.cc:1918] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:20.918851 3750 log.cc:1196] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/d68f9125004f40dbaf9f32d3e127f540
I20260811 02:47:20.919165 3750 ts_tablet_manager.cc:1939] T d68f9125004f40dbaf9f32d3e127f540 P 75f5fd4a5c33444da5694e653491cdd1: Deleting consensus metadata
I20260811 02:47:20.919874 2756 catalog_manager.cc:4841] TS 75f5fd4a5c33444da5694e653491cdd1 (127.21.236.130:43023): tablet d68f9125004f40dbaf9f32d3e127f540 (table test-table [id=1091f7f428614735beea6a655c181df5]) successfully deleted
I20260811 02:47:20.920163 3298 tablet_service.cc:1467] Processing CreateTablet for tablet 2727c6814b83471ca18507f7c00f5a6a (DEFAULT_TABLE table=test-table [id=be2d4d9613d44dfe855272e9f3998f9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:20.920233 3038 tablet_service.cc:1467] Processing CreateTablet for tablet 2727c6814b83471ca18507f7c00f5a6a (DEFAULT_TABLE table=test-table [id=be2d4d9613d44dfe855272e9f3998f9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:20.920449 3298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2727c6814b83471ca18507f7c00f5a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:20.920521 3038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2727c6814b83471ca18507f7c00f5a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:20.921017 3547 tablet_service.cc:1467] Processing CreateTablet for tablet 2727c6814b83471ca18507f7c00f5a6a (DEFAULT_TABLE table=test-table [id=be2d4d9613d44dfe855272e9f3998f9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:20.921262 3547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2727c6814b83471ca18507f7c00f5a6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:20.922741 3706 tablet_bootstrap.cc:492] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1: Bootstrap starting.
I20260811 02:47:20.923700 3711 tablet_bootstrap.cc:492] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f: Bootstrap starting.
I20260811 02:47:20.923754 3706 tablet_bootstrap.cc:654] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:20.924431 3712 tablet_bootstrap.cc:492] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025: Bootstrap starting.
I20260811 02:47:20.924553 3711 tablet_bootstrap.cc:654] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:20.925297 3712 tablet_bootstrap.cc:654] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:20.925657 3711 tablet_bootstrap.cc:492] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f: No bootstrap required, opened a new log
I20260811 02:47:20.925673 3706 tablet_bootstrap.cc:492] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1: No bootstrap required, opened a new log
I20260811 02:47:20.925738 3706 ts_tablet_manager.cc:1397] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:47:20.925740 3711 ts_tablet_manager.cc:1397] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:47:20.926144 3711 raft_consensus.cc:348] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.926144 3706 raft_consensus.cc:348] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.926255 3711 raft_consensus.cc:374] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:20.926255 3706 raft_consensus.cc:374] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:20.926290 3706 raft_consensus.cc:729] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75f5fd4a5c33444da5694e653491cdd1, State: Initialized, Role: FOLLOWER
I20260811 02:47:20.926290 3711 raft_consensus.cc:729] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6c99522e0164493ab63bd6137c8e85f, State: Initialized, Role: FOLLOWER
I20260811 02:47:20.926406 3712 tablet_bootstrap.cc:492] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025: No bootstrap required, opened a new log
I20260811 02:47:20.926404 3711 consensus_queue.cc:260] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.926400 3706 consensus_queue.cc:260] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.926491 3712 ts_tablet_manager.cc:1397] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:47:20.926662 3711 ts_tablet_manager.cc:1428] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 02:47:20.926657 3706 ts_tablet_manager.cc:1428] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:47:20.926872 3712 raft_consensus.cc:348] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.927026 3712 raft_consensus.cc:374] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:20.928254 3712 raft_consensus.cc:729] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65c1aaacde848ebaf5ce2df1b01e025, State: Initialized, Role: FOLLOWER
I20260811 02:47:20.928406 3712 consensus_queue.cc:260] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:20.928710 3712 ts_tablet_manager.cc:1428] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 02:47:21.104365 3716 raft_consensus.cc:482] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:47:21.104525 3716 raft_consensus.cc:504] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:21.105748 3716 leader_election.cc:288] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65c1aaacde848ebaf5ce2df1b01e025 (127.21.236.129:39753), e6c99522e0164493ab63bd6137c8e85f (127.21.236.131:40905)
I20260811 02:47:21.111120 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:21.106210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:53993 (local address 127.21.236.131:40905)
0811 02:47:21.106466 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:21.106471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:21.106576 (+ 105us) server_negotiation.cc:407] Connection header received
0811 02:47:21.107014 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:21.107017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:21.107106 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:21.107193 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:21.107433 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.108176 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.109772 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.110380 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.110507 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:21.110587 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:21.110894 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 02:47:21.110969 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260811 02:47:21.111120 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:21.106202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:39723 (local address 127.21.236.129:39753)
0811 02:47:21.106348 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:21.106354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:21.106507 (+ 153us) server_negotiation.cc:407] Connection header received
0811 02:47:21.106759 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:21.106762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:21.106826 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:21.106896 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:21.107238 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.108033 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.109772 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.110387 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.110507 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:21.110565 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:21.110894 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 02:47:21.110969 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 02:47:21.111196 3849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:21.106062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:39753 (local address 127.21.236.130:39723)
0811 02:47:21.106403 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:21.106420 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:21.106641 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:21.106923 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:21.106925 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:21.106941 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:21.107058 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:21.107084 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.108156 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.108158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:21.109643 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:21.109651 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.110529 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.110532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:21.110724 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:21.110780 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 02:47:21.110861 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 02:47:21.110955 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":145,"spinlock_wait_cycles":20096,"thread_start_us":88,"threads_started":1}
I20260811 02:47:21.111387 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:21.106062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:40905 (local address 127.21.236.130:53993)
0811 02:47:21.106477 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:21.106491 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:21.106641 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:21.107211 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:21.107217 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:21.107236 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:21.107331 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:21.107335 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.108295 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.108297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:21.109639 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:21.109651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.110529 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.110532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:21.110721 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:21.110788 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 02:47:21.110861 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 02:47:21.110955 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":200,"spinlock_wait_cycles":72960,"thread_start_us":67,"threads_started":1}
I20260811 02:47:21.111755 3079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2727c6814b83471ca18507f7c00f5a6a" candidate_uuid: "75f5fd4a5c33444da5694e653491cdd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" is_pre_election: true
I20260811 02:47:21.111909 3079 raft_consensus.cc:2454] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f5fd4a5c33444da5694e653491cdd1 in term 0.
I20260811 02:47:21.111912 3575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2727c6814b83471ca18507f7c00f5a6a" candidate_uuid: "75f5fd4a5c33444da5694e653491cdd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6c99522e0164493ab63bd6137c8e85f" is_pre_election: true
I20260811 02:47:21.112079 3575 raft_consensus.cc:2454] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75f5fd4a5c33444da5694e653491cdd1 in term 0.
I20260811 02:47:21.112298 3210 leader_election.cc:302] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f5fd4a5c33444da5694e653491cdd1, e65c1aaacde848ebaf5ce2df1b01e025; no voters:
I20260811 02:47:21.112569 3716 raft_consensus.cc:2781] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:47:21.112632 3716 raft_consensus.cc:482] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:47:21.112671 3716 raft_consensus.cc:3037] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:21.113641 3716 raft_consensus.cc:504] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:21.113978 3716 leader_election.cc:288] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [CANDIDATE]: Term 1 election: Requested vote from peers e65c1aaacde848ebaf5ce2df1b01e025 (127.21.236.129:39753), e6c99522e0164493ab63bd6137c8e85f (127.21.236.131:40905)
I20260811 02:47:21.114315 3079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2727c6814b83471ca18507f7c00f5a6a" candidate_uuid: "75f5fd4a5c33444da5694e653491cdd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65c1aaacde848ebaf5ce2df1b01e025"
I20260811 02:47:21.114380 3575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2727c6814b83471ca18507f7c00f5a6a" candidate_uuid: "75f5fd4a5c33444da5694e653491cdd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6c99522e0164493ab63bd6137c8e85f"
I20260811 02:47:21.114432 3079 raft_consensus.cc:3037] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:21.114526 3575 raft_consensus.cc:3037] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:21.115514 3079 raft_consensus.cc:2454] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f5fd4a5c33444da5694e653491cdd1 in term 1.
I20260811 02:47:21.115547 3575 raft_consensus.cc:2454] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75f5fd4a5c33444da5694e653491cdd1 in term 1.
I20260811 02:47:21.115854 3208 leader_election.cc:302] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75f5fd4a5c33444da5694e653491cdd1, e6c99522e0164493ab63bd6137c8e85f; no voters:
I20260811 02:47:21.116020 3716 raft_consensus.cc:2781] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:21.116216 3716 raft_consensus.cc:686] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [term 1 LEADER]: Becoming Leader. State: Replica: 75f5fd4a5c33444da5694e653491cdd1, State: Running, Role: LEADER
I20260811 02:47:21.116482 3716 consensus_queue.cc:237] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } }
I20260811 02:47:21.118640 2784 catalog_manager.cc:5495] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75f5fd4a5c33444da5694e653491cdd1 (127.21.236.130). New cstate: current_term: 1 leader_uuid: "75f5fd4a5c33444da5694e653491cdd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75f5fd4a5c33444da5694e653491cdd1" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 43023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 } health_report { overall_health: UNKNOWN } } }
I20260811 02:47:21.175595 3852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:21.171918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41226 (local address 127.21.236.130:43023)
0811 02:47:21.172199 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:21.172206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:21.172217 (+ 11us) server_negotiation.cc:407] Connection header received
0811 02:47:21.172287 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:21.172289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:21.172340 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:21.172444 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:47:21.172663 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.173177 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.173957 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.174188 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.174267 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:21.174482 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:47:21.174994 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:47:21.175138 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:21.175301 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 02:47:21.175379 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":131,"threads_started":1}
I20260811 02:47:21.186344 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:21.183147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:40199 (local address 127.21.236.131:40905)
0811 02:47:21.183308 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:21.183312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:21.183429 (+ 117us) server_negotiation.cc:407] Connection header received
0811 02:47:21.183539 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:21.183542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:21.183590 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:21.183657 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:47:21.183993 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.184496 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.185380 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.185610 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.185691 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:21.185959 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:47:21.186032 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:47:21.186087 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:21.186191 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 02:47:21.186239 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:47:21.186360 3859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:21.182972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:40905 (local address 127.21.236.130:40199)
0811 02:47:21.183328 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:21.183341 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:21.183435 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:21.183722 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:21.183724 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:21.183732 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:47:21.183848 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:21.183852 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.184604 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.184607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:21.185278 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:21.185286 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.185715 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.185717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:21.185790 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:21.185834 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:47:21.186109 (+ 275us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:47:21.186114 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 02:47:21.186165 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 02:47:21.186228 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":141,"threads_started":1}
I20260811 02:47:21.193073 3079 raft_consensus.cc:1261] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Refusing update from remote peer 75f5fd4a5c33444da5694e653491cdd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:21.193125 3575 raft_consensus.cc:1261] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Refusing update from remote peer 75f5fd4a5c33444da5694e653491cdd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:21.193549 3716 consensus_queue.cc:1035] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65c1aaacde848ebaf5ce2df1b01e025" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 39753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:21.193995 3850 consensus_queue.cc:1035] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6c99522e0164493ab63bd6137c8e85f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 40905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:21.210569 3869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:21.207121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:43023 (local address 127.21.236.131:43299)
0811 02:47:21.207411 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:21.207427 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:21.207529 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:21.207809 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:21.207811 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:21.207819 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 02:47:21.207960 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:21.207966 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.208673 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.208675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:21.209336 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:21.209346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.209857 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.209861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:21.209951 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:21.209987 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 02:47:21.210276 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 02:47:21.210285 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 02:47:21.210350 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 02:47:21.210436 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":59,"threads_started":1}
I20260811 02:47:21.210544 3852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:21.207178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:43299 (local address 127.21.236.130:43023)
0811 02:47:21.207281 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:21.207286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:21.207490 (+ 204us) server_negotiation.cc:407] Connection header received
0811 02:47:21.207652 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:21.207655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:21.207703 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:21.207795 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:47:21.208070 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.208556 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.209457 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:21.209752 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:21.209830 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:21.210119 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:47:21.210183 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:47:21.210267 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:21.210359 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 02:47:21.210409 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 02:47:21.234499 2784 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:21.234679 2784 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:21.236845 2784 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de51c0fa70c344e0a22dab0e6c92f2e3: Sending DeleteTablet for 3 replicas of tablet 2727c6814b83471ca18507f7c00f5a6a
I20260811 02:47:21.237399 3298 tablet_service.cc:1514] Processing DeleteTablet for tablet 2727c6814b83471ca18507f7c00f5a6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:21 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:41220
I20260811 02:47:21.237399 3038 tablet_service.cc:1514] Processing DeleteTablet for tablet 2727c6814b83471ca18507f7c00f5a6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:21 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:44554
I20260811 02:47:21.237635 3750 tablet_replica.cc:330] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1: stopping tablet replica
I20260811 02:47:21.237768 3750 raft_consensus.cc:2229] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:47:21.237792 3547 tablet_service.cc:1514] Processing DeleteTablet for tablet 2727c6814b83471ca18507f7c00f5a6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:21 UTC) from {username='oryx', principal='oryx/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:41190
I20260811 02:47:21.238009 3752 tablet_replica.cc:330] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f: stopping tablet replica
I20260811 02:47:21.238075 3750 raft_consensus.cc:2258] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:21.238133 3752 raft_consensus.cc:2229] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:21.238245 3752 raft_consensus.cc:2258] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:21.238265 3753 tablet_replica.cc:330] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025: stopping tablet replica
I20260811 02:47:21.238384 3753 raft_consensus.cc:2229] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:21.238489 3753 raft_consensus.cc:2258] T 2727c6814b83471ca18507f7c00f5a6a P e65c1aaacde848ebaf5ce2df1b01e025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:21.238793 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 2923
I20260811 02:47:21.242765 3752 ts_tablet_manager.cc:1905] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:21.246238 3752 ts_tablet_manager.cc:1918] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:47:21.246356 3752 log.cc:1196] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/2727c6814b83471ca18507f7c00f5a6a
I20260811 02:47:21.246629 3752 ts_tablet_manager.cc:1939] T 2727c6814b83471ca18507f7c00f5a6a P e6c99522e0164493ab63bd6137c8e85f: Deleting consensus metadata
I20260811 02:47:21.247354 2761 catalog_manager.cc:4841] TS e6c99522e0164493ab63bd6137c8e85f (127.21.236.131:40905): tablet 2727c6814b83471ca18507f7c00f5a6a (table test-table [id=be2d4d9613d44dfe855272e9f3998f9b]) successfully deleted
I20260811 02:47:21.248703 3750 ts_tablet_manager.cc:1905] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:21.250993 3750 ts_tablet_manager.cc:1918] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 02:47:21.251101 3750 log.cc:1196] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/2727c6814b83471ca18507f7c00f5a6a
I20260811 02:47:21.251374 3750 ts_tablet_manager.cc:1939] T 2727c6814b83471ca18507f7c00f5a6a P 75f5fd4a5c33444da5694e653491cdd1: Deleting consensus metadata
I20260811 02:47:21.252034 2756 catalog_manager.cc:4841] TS 75f5fd4a5c33444da5694e653491cdd1 (127.21.236.130:43023): tablet 2727c6814b83471ca18507f7c00f5a6a (table test-table [id=be2d4d9613d44dfe855272e9f3998f9b]) successfully deleted
W20260811 02:47:21.260697 2763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:39753 (error 108)
I20260811 02:47:21.260907 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 3177
I20260811 02:47:21.271787 3875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:21.271360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:39753 (local address 127.0.0.1:44586)
0811 02:47:21.271619 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:21.271696 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:39753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
W20260811 02:47:21.272064 2763 catalog_manager.cc:4584] TS e65c1aaacde848ebaf5ce2df1b01e025 (127.21.236.129:39753): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2727c6814b83471ca18507f7c00f5a6a: Network error: Client connection negotiation failed: client connection to 127.21.236.129:39753: connect: Connection refused (error 111)
I20260811 02:47:21.276789 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 3441
I20260811 02:47:21.294569 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 2729
2026-08-11T02:47:21Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3920 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:47:21 dist-test-slave-n326 krb5kdc[3881](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:47:21 dist-test-slave-n326 krb5kdc[3881](info): set up 2 sockets
Aug 11 02:47:21 dist-test-slave-n326 krb5kdc[3881](info): commencing operation
krb5kdc: starting...
W20260811 02:47:22.634169 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:47:22 dist-test-slave-n326 krb5kdc[3881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:47:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:47:22.769801 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:45113
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:45113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:22.882248 3896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:22.882532 3896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:22.882609 3896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:22.886600 3896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:22.886687 3896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:22.886724 3896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:22.886761 3896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:22.886792 3896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:47:22.891993 3896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:45113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:45113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:22.893251 3896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:22.894380 3896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:22.903915 3909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:22.903928 3908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:22.903962 3911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:22.903942 3896 server_base.cc:1034] running on GCE node
I20260811 02:47:22.904690 3896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:22.905751 3896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:22.906966 3896 hybrid_clock.cc:648] HybridClock initialized: now 1786416442906931 us; error 33 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T02:47:22Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1611 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:47:22 dist-test-slave-n326 krb5kdc[3917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:47:22 dist-test-slave-n326 krb5kdc[3917](info): set up 2 sockets
Aug 11 02:47:22 dist-test-slave-n326 krb5kdc[3917](info): commencing operation
krb5kdc: starting...
W20260811 02:47:24.195749 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:47:24 dist-test-slave-n326 krb5kdc[3917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:47:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:47:24.332589 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:40517
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:45229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:40517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:24.446535 3939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:24.446806 3939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:24.446847 3939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:24.450964 3939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:24.451040 3939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:24.451076 3939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:24.451092 3939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:24.451108 3939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:47:24.456427 3939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:40517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:40517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:24.457486 3939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:24.458508 3939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:24.467594 3949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:24.467775 3953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:24.467614 3956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:24.468319 3939 server_base.cc:1034] running on GCE node
I20260811 02:47:24.468997 3939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:24.470534 3939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:24.471753 3939 hybrid_clock.cc:648] HybridClock initialized: now 1786416444471675 us; error 91 us; skew 500 ppm
Aug 11 02:47:24 dist-test-slave-n326 krb5kdc[3917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416444, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:24.475468 3939 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:47:24.476627 3939 webserver.cc:460] Webserver started at http://127.21.236.190:34985/ using document root <none> and password file <none>
I20260811 02:47:24.477385 3939 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:24.477488 3939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:24.477761 3939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:24.480741 3939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "6e0dd6e0095b43d6ac9af4eb063e469b"
format_stamp: "Formatted at 2026-08-11 02:47:24 on dist-test-slave-n326"
server_key: "3b4b7ba5a65c60a7131b038600614ea3"
server_key_iv: "c7c8a1ab75ab83be2e6108135bc610c3"
server_key_version: "encryptionkey@0"
I20260811 02:47:24.481472 3939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "6e0dd6e0095b43d6ac9af4eb063e469b"
format_stamp: "Formatted at 2026-08-11 02:47:24 on dist-test-slave-n326"
server_key: "3b4b7ba5a65c60a7131b038600614ea3"
server_key_iv: "c7c8a1ab75ab83be2e6108135bc610c3"
server_key_version: "encryptionkey@0"
I20260811 02:47:24.488217 3939 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 02:47:24.491539 3970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:24.493225 3939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:47:24.493372 3939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "6e0dd6e0095b43d6ac9af4eb063e469b"
format_stamp: "Formatted at 2026-08-11 02:47:24 on dist-test-slave-n326"
server_key: "3b4b7ba5a65c60a7131b038600614ea3"
server_key_iv: "c7c8a1ab75ab83be2e6108135bc610c3"
server_key_version: "encryptionkey@0"
I20260811 02:47:24.493500 3939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:24.508172 3939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:24.512305 3939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:24.512528 3939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:24.523000 3939 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:40517
I20260811 02:47:24.523051 4051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:40517 every 8 connection(s)
I20260811 02:47:24.524027 3939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:24.526810 4052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:24.529948 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 3939
I20260811 02:47:24.530109 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:47:24.530387 22450 external_mini_cluster.cc:1428] Setting key 1161518f8c764a8d393129ac2a4b6489
I20260811 02:47:24.532866 4052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b: Bootstrap starting.
I20260811 02:47:24.535141 4052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:24.535887 4052 log.cc:824] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:24.538244 4052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b: No bootstrap required, opened a new log
Aug 11 02:47:24 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:24.542606 4052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40517 } }
I20260811 02:47:24.542876 4052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:24.542964 4052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0dd6e0095b43d6ac9af4eb063e469b, State: Initialized, Role: FOLLOWER
I20260811 02:47:24.543409 4052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40517 } }
I20260811 02:47:24.543542 4052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:24.543625 4052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:24.543730 4052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:24.544667 4052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40517 } }
I20260811 02:47:24.545022 4052 leader_election.cc:302] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e0dd6e0095b43d6ac9af4eb063e469b; no voters:
I20260811 02:47:24.545321 4052 leader_election.cc:288] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:47:24.545426 4066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:24.545635 4066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 1 LEADER]: Becoming Leader. State: Replica: 6e0dd6e0095b43d6ac9af4eb063e469b, State: Running, Role: LEADER
I20260811 02:47:24.545750 4061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:24.531836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41968 (local address 127.21.236.190:40517)
0811 02:47:24.532271 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:24.532285 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:24.532320 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:47:24.532937 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:24.532953 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:24.533321 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:24.533650 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:24.534458 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:24.535583 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:24.536661 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:24.537347 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:24.537535 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:24.540364 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:24.540369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:24.540382 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:24.540408 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:24.542243 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:24.542754 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:24.542759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:24.542763 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:24.542917 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:24.543384 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:24.543387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:24.543389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:24.543953 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:24.544172 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:24.544462 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 02:47:24.544782 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":142,"threads_started":1}
I20260811 02:47:24.545948 4066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40517 } }
I20260811 02:47:24.546619 4052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:24.547228 4066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e0dd6e0095b43d6ac9af4eb063e469b. Latest consensus state: current_term: 1 leader_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40517 } } }
I20260811 02:47:24.547345 4066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:24.547947 4068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40517 } } }
I20260811 02:47:24.548058 4068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:24.549263 4094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:24.552438 4094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:24.557745 4094 catalog_manager.cc:1347] Generated new cluster ID: 2c7770f4e039403eae0361ab7272c372
I20260811 02:47:24.557811 4094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:24.576041 4094 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:47:24.577075 4094 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:24.587824 4094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b: Generated new TSK 0
I20260811 02:47:24.588526 4094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:47:24.629866 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:40517
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:45229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:24.748803 4110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:24.749107 4110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:24.749183 4110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:24.753295 4110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:24.753388 4110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:24.753479 4110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:47:24.758862 4110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:40517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:24.760046 4110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:24.761142 4110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:24.771054 4132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:24.771098 4135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:24.771090 4130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:24.771413 4110 server_base.cc:1034] running on GCE node
I20260811 02:47:24.771934 4110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:24.772706 4110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:24.773943 4110 hybrid_clock.cc:648] HybridClock initialized: now 1786416444773912 us; error 56 us; skew 500 ppm
Aug 11 02:47:24 dist-test-slave-n326 krb5kdc[3917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416444, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:24.776954 4110 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:47:24.777889 4110 webserver.cc:460] Webserver started at http://127.21.236.129:33759/ using document root <none> and password file <none>
I20260811 02:47:24.778497 4110 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:24.778625 4110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:24.778880 4110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:24.781141 4110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "7ffa036cae64465eaaa6468eb93fbb53"
format_stamp: "Formatted at 2026-08-11 02:47:24 on dist-test-slave-n326"
server_key: "fedb4b70526c5f6973d62ec2edb950b7"
server_key_iv: "1db4633ef9fa6f35f7adf67517fd6963"
server_key_version: "encryptionkey@0"
I20260811 02:47:24.781786 4110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "7ffa036cae64465eaaa6468eb93fbb53"
format_stamp: "Formatted at 2026-08-11 02:47:24 on dist-test-slave-n326"
server_key: "fedb4b70526c5f6973d62ec2edb950b7"
server_key_iv: "1db4633ef9fa6f35f7adf67517fd6963"
server_key_version: "encryptionkey@0"
I20260811 02:47:24.785871 4110 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:47:24.788626 4145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:24.789901 4110 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:47:24.790089 4110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "7ffa036cae64465eaaa6468eb93fbb53"
format_stamp: "Formatted at 2026-08-11 02:47:24 on dist-test-slave-n326"
server_key: "fedb4b70526c5f6973d62ec2edb950b7"
server_key_iv: "1db4633ef9fa6f35f7adf67517fd6963"
server_key_version: "encryptionkey@0"
I20260811 02:47:24.790217 4110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:24.804698 4110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:24.808199 4110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:24.808423 4110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:24.809056 4110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:24.809942 4110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:24.810016 4110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:24.810097 4110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:24.810141 4110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:24.819291 4110 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:37161
I20260811 02:47:24.819335 4358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:37161 every 8 connection(s)
I20260811 02:47:24.820526 4110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:47:24.827445 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 4110
I20260811 02:47:24.827626 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:47:24.827934 22450 external_mini_cluster.cc:1428] Setting key d4f1615a7846754359fc04e8c7937a9d
Aug 11 02:47:24 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416444, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:24.835583 4061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:24.822514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:56363 (local address 127.21.236.190:40517)
0811 02:47:24.822660 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:24.822664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:24.823507 (+ 843us) server_negotiation.cc:407] Connection header received
0811 02:47:24.824890 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:24.824893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:24.824954 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:24.825036 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:24.826226 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:24.826785 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:24.828163 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:24.828501 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:24.828610 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:24.831405 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:24.831410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:24.831414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:24.831445 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:24.832933 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:24.833672 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:24.833675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:24.833676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:24.833768 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:24.834273 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:24.834276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:24.834278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:24.834569 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:24.834682 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:24.835326 (+ 644us) server_negotiation.cc:299] Negotiation successful
0811 02:47:24.835452 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:47:24.837442 4362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:24.822898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:56363)
0811 02:47:24.823353 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:24.823395 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:24.824671 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:24.825217 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:24.825224 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:24.825593 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:24.826066 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:24.826080 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:24.826947 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:24.826950 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:24.828002 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:24.828036 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:24.828659 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:24.828663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:24.828874 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:24.829656 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:24.829674 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:24.831231 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:24.833107 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:24.833117 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:24.833137 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:24.833518 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:24.833875 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:24.833881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:24.833886 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:24.834121 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:24.834681 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:24.834690 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:24.834975 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 02:47:24.835845 (+ 870us) client_negotiation.cc:240] Negotiation successful
0811 02:47:24.836281 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":133,"threads_started":1}
I20260811 02:47:24.838938 4359 heartbeater.cc:347] Connected to a master server at 127.21.236.190:40517
I20260811 02:47:24.839231 4359 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:24.839762 4359 heartbeater.cc:511] Master 127.21.236.190:40517 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
I20260811 02:47:24.841483 4000 ts_manager.cc:194] Registered new tserver with Master: 7ffa036cae64465eaaa6468eb93fbb53 (127.21.236.129:37161)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
I20260811 02:47:24.842913 4000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:56363
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:47:24.862692 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:40517
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:45229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:24.978044 4365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:24.978387 4365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:24.978432 4365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:24.982611 4365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:24.982684 4365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:24.982743 4365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:47:24.988073 4365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:40517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:24.989182 4365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:24.990170 4365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:25.000212 4376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:25.000233 4373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:25.000550 4365 server_base.cc:1034] running on GCE node
W20260811 02:47:25.000213 4374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:25.000970 4365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:25.001621 4365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:25.002795 4365 hybrid_clock.cc:648] HybridClock initialized: now 1786416445002766 us; error 43 us; skew 500 ppm
Aug 11 02:47:25 dist-test-slave-n326 krb5kdc[3917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:25.005868 4365 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:47:25.006794 4365 webserver.cc:460] Webserver started at http://127.21.236.130:43113/ using document root <none> and password file <none>
I20260811 02:47:25.007474 4365 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:25.007576 4365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:25.007807 4365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:25.009961 4365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "6ed7cf0f74d540b0a4b295753ecf1bd8"
format_stamp: "Formatted at 2026-08-11 02:47:25 on dist-test-slave-n326"
server_key: "992aeebf69c879e1efac2fff73706696"
server_key_iv: "a015d92d6b654f788a04f75ee41bc064"
server_key_version: "encryptionkey@0"
I20260811 02:47:25.010526 4365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "6ed7cf0f74d540b0a4b295753ecf1bd8"
format_stamp: "Formatted at 2026-08-11 02:47:25 on dist-test-slave-n326"
server_key: "992aeebf69c879e1efac2fff73706696"
server_key_iv: "a015d92d6b654f788a04f75ee41bc064"
server_key_version: "encryptionkey@0"
I20260811 02:47:25.014243 4365 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 02:47:25.016765 4386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:25.017971 4365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:47:25.018108 4365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "6ed7cf0f74d540b0a4b295753ecf1bd8"
format_stamp: "Formatted at 2026-08-11 02:47:25 on dist-test-slave-n326"
server_key: "992aeebf69c879e1efac2fff73706696"
server_key_iv: "a015d92d6b654f788a04f75ee41bc064"
server_key_version: "encryptionkey@0"
I20260811 02:47:25.018216 4365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:25.028810 4365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:25.032088 4365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:25.032291 4365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:25.032878 4365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:25.033720 4365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:25.033792 4365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:25.033864 4365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:25.033917 4365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:25.043510 4365 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:32905
I20260811 02:47:25.043553 4591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:32905 every 8 connection(s)
I20260811 02:47:25.044605 4365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:47:25.050163 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 4365
I20260811 02:47:25.050407 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:47:25.050784 22450 external_mini_cluster.cc:1428] Setting key b300c49543e253cbc58605d5595a4cbc
Aug 11 02:47:25 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:25.060446 4061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:25.046352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:51771 (local address 127.21.236.190:40517)
0811 02:47:25.046503 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:25.046507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:25.047482 (+ 975us) server_negotiation.cc:407] Connection header received
0811 02:47:25.049108 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:25.049111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:25.049170 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:25.049279 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:25.050487 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.051247 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.052854 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.053240 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.053364 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:25.056321 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:25.056327 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:25.056333 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:25.056373 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:25.058139 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:25.058688 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:25.058695 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:25.058700 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:25.058802 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:25.059223 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:25.059229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:25.059232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:25.059501 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:25.059619 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:25.060122 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 02:47:25.060257 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:47:25.061223 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:25.046703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:51771)
0811 02:47:25.047335 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:25.047379 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:25.048894 (+ 1515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:25.049473 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:25.049479 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:25.049826 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:25.050311 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:25.050324 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.051394 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.051398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:25.052720 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:25.052732 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.053406 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.053410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:25.053603 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:25.054280 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:25.054303 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:25.056155 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:25.058274 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:25.058278 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:25.058290 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:25.058571 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:25.058907 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:25.058909 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:25.058910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:25.059114 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:25.059613 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:25.059618 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:25.059848 (+ 230us) client_negotiation.cc:769] Sending connection context
0811 02:47:25.060003 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 02:47:25.060269 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":120,"threads_started":1}
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
I20260811 02:47:25.062438 4592 heartbeater.cc:347] Connected to a master server at 127.21.236.190:40517
I20260811 02:47:25.062707 4592 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:25.063261 4592 heartbeater.cc:511] Master 127.21.236.190:40517 requested a full tablet report, sending...
I20260811 02:47:25.064499 4000 ts_manager.cc:194] Registered new tserver with Master: 6ed7cf0f74d540b0a4b295753ecf1bd8 (127.21.236.130:32905)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
I20260811 02:47:25.065196 4000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:51771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:47:25.081779 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:40517
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:45229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:25.193785 4601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:25.194123 4601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:25.194202 4601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:25.198328 4601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:25.198382 4601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:25.198438 4601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:47:25.203842 4601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:40517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:25.204913 4601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:25.205924 4601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:47:25.215900 4601 server_base.cc:1034] running on GCE node
W20260811 02:47:25.215837 4612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:25.215873 4616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:25.215833 4611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:25.216702 4601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:25.217389 4601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:25.218636 4601 hybrid_clock.cc:648] HybridClock initialized: now 1786416445218610 us; error 49 us; skew 500 ppm
Aug 11 02:47:25 dist-test-slave-n326 krb5kdc[3917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:25.221753 4601 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:47:25.222777 4601 webserver.cc:460] Webserver started at http://127.21.236.131:36887/ using document root <none> and password file <none>
I20260811 02:47:25.223397 4601 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:25.223464 4601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:25.223634 4601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:25.225632 4601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "041aed63510e473a83b151b43eaebe26"
format_stamp: "Formatted at 2026-08-11 02:47:25 on dist-test-slave-n326"
server_key: "880903d6d863ffc5b5269fb4e7361cb8"
server_key_iv: "9b0990f95a3b4d6f227dd92caa6dacab"
server_key_version: "encryptionkey@0"
I20260811 02:47:25.226150 4601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "041aed63510e473a83b151b43eaebe26"
format_stamp: "Formatted at 2026-08-11 02:47:25 on dist-test-slave-n326"
server_key: "880903d6d863ffc5b5269fb4e7361cb8"
server_key_iv: "9b0990f95a3b4d6f227dd92caa6dacab"
server_key_version: "encryptionkey@0"
I20260811 02:47:25.230129 4601 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:47:25.232689 4633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:25.233942 4601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:47:25.234053 4601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "041aed63510e473a83b151b43eaebe26"
format_stamp: "Formatted at 2026-08-11 02:47:25 on dist-test-slave-n326"
server_key: "880903d6d863ffc5b5269fb4e7361cb8"
server_key_iv: "9b0990f95a3b4d6f227dd92caa6dacab"
server_key_version: "encryptionkey@0"
I20260811 02:47:25.234195 4601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:25.279377 4601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:25.282706 4601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:25.282922 4601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:25.283591 4601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:25.284456 4601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:25.284503 4601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:25.284574 4601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:25.284608 4601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:25.293788 4601 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:46539
I20260811 02:47:25.293819 4839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:46539 every 8 connection(s)
I20260811 02:47:25.294773 4601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:47:25.299091 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 4601
I20260811 02:47:25.299237 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:47:25.299593 22450 external_mini_cluster.cc:1428] Setting key a22329fcf249d5ef9f0cb59ecd1c3692
Aug 11 02:47:25 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:25.309087 4061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:25.296758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:49529 (local address 127.21.236.190:40517)
0811 02:47:25.296959 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:25.296964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:25.297885 (+ 921us) server_negotiation.cc:407] Connection header received
0811 02:47:25.299425 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:25.299428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:25.299480 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:25.299562 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:25.301074 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.301616 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.302839 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.303125 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.303236 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:25.305732 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:25.305735 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:25.305737 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:25.305762 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:25.307057 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:25.307664 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:25.307666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:25.307667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:25.307743 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:25.308068 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:25.308070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:25.308071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:25.308239 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:25.308316 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:25.308854 (+ 538us) server_negotiation.cc:299] Negotiation successful
0811 02:47:25.308950 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 02:47:25.310065 4844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:25.297227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:49529)
0811 02:47:25.297726 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:25.297769 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:25.299186 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:25.299772 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:25.299781 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:25.300232 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:25.300919 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:25.300937 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.301744 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.301748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:25.302703 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:25.302714 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.303276 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.303280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:25.303463 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:25.304084 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:25.304101 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:25.305577 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:25.307218 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:25.307223 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:25.307235 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:25.307570 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:25.307822 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:25.307824 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:25.307825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:25.307977 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:25.308358 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:25.308364 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:25.308632 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 02:47:25.308813 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 02:47:25.309121 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":127,"threads_started":1}
I20260811 02:47:25.311255 4841 heartbeater.cc:347] Connected to a master server at 127.21.236.190:40517
I20260811 02:47:25.311508 4841 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:25.312014 4841 heartbeater.cc:511] Master 127.21.236.190:40517 requested a full tablet report, sending...
I20260811 02:47:25.313181 4000 ts_manager.cc:194] Registered new tserver with Master: 041aed63510e473a83b151b43eaebe26 (127.21.236.131:46539)
I20260811 02:47:25.313675 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:47:25.313969 4000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:49529
WARNING: no policy specified for oryx/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.129.keytab.
Entry for principal oryx/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.129.keytab.
WARNING: no policy specified for oryx/127.21.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.130.keytab.
Entry for principal oryx/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.130.keytab.
WARNING: no policy specified for oryx/127.21.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.131.keytab.
Entry for principal oryx/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.131.keytab.
I20260811 02:47:25.397964 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 4110
I20260811 02:47:25.414528 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 4365
I20260811 02:47:25.426440 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 4601
I20260811 02:47:25.439579 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 3939
I20260811 02:47:25.453202 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:40517
--webserver_interface=127.21.236.190
--webserver_port=34985
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:45229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:40517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:25.564051 4859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:25.564314 4859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:25.564354 4859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:25.568374 4859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:25.568430 4859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:25.568449 4859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:25.568467 4859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:25.568485 4859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:47:25.573707 4859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:40517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:40517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=34985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:25.574754 4859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:25.575781 4859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:25.584339 4881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:25.584334 4877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:25.584334 4875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:25.584568 4859 server_base.cc:1034] running on GCE node
I20260811 02:47:25.585161 4859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:25.586301 4859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:25.587493 4859 hybrid_clock.cc:648] HybridClock initialized: now 1786416445587480 us; error 34 us; skew 500 ppm
Aug 11 02:47:25 dist-test-slave-n326 krb5kdc[3917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416445, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:25.590247 4859 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:47:25.591240 4859 webserver.cc:460] Webserver started at http://127.21.236.190:34985/ using document root <none> and password file <none>
I20260811 02:47:25.591810 4859 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:25.591928 4859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:25.595679 4859 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 02:47:25.597856 4893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:25.599134 4859 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:47:25.599268 4859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "6e0dd6e0095b43d6ac9af4eb063e469b"
format_stamp: "Formatted at 2026-08-11 02:47:24 on dist-test-slave-n326"
server_key: "3b4b7ba5a65c60a7131b038600614ea3"
server_key_iv: "c7c8a1ab75ab83be2e6108135bc610c3"
server_key_version: "encryptionkey@0"
I20260811 02:47:25.599836 4859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:25.613036 4859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:25.615782 4859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:25.615988 4859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:25.623416 4859 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:40517
I20260811 02:47:25.623442 4975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:40517 every 8 connection(s)
I20260811 02:47:25.624466 4859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:25.628708 4976 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:47:25.629877 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 4859
I20260811 02:47:25.630354 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:37161
--local_ip_for_outbound_sockets=127.21.236.129
--tserver_master_addrs=127.21.236.190:40517
--webserver_port=33759
--webserver_interface=127.21.236.129
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:45229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.236.129
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:47:25.630404 4976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b: Bootstrap starting.
I20260811 02:47:25.653663 4976 log.cc:824] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:25.662055 4976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:47:25.662479 4976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b: Bootstrap complete.
I20260811 02:47:25.668280 4976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40517 } }
I20260811 02:47:25.668810 4976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0dd6e0095b43d6ac9af4eb063e469b, State: Initialized, Role: FOLLOWER
I20260811 02:47:25.669344 4976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40517 } }
I20260811 02:47:25.669487 4976 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:25.669554 4976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:25.669639 4976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:47:25.672112 4976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40517 } }
I20260811 02:47:25.672469 4976 leader_election.cc:302] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e0dd6e0095b43d6ac9af4eb063e469b; no voters:
I20260811 02:47:25.672818 4976 leader_election.cc:288] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:47:25.672967 4985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:47:25.673259 4985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [term 2 LEADER]: Becoming Leader. State: Replica: 6e0dd6e0095b43d6ac9af4eb063e469b, State: Running, Role: LEADER
I20260811 02:47:25.673569 4985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40517 } }
I20260811 02:47:25.674046 4976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:25.675200 4987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40517 } } }
I20260811 02:47:25.675316 4987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:25.675172 4988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e0dd6e0095b43d6ac9af4eb063e469b. Latest consensus state: current_term: 2 leader_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e0dd6e0095b43d6ac9af4eb063e469b" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40517 } } }
I20260811 02:47:25.675418 4988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:25.675729 4999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:25.679379 4999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:25.680025 4999 catalog_manager.cc:1259] Loaded cluster ID: 2c7770f4e039403eae0361ab7272c372
I20260811 02:47:25.680104 4999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:25.682473 4999 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:25.683171 4999 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6e0dd6e0095b43d6ac9af4eb063e469b: Loaded TSK: 0
I20260811 02:47:25.684383 4999 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 02:47:25.762837 4980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:25.763126 4980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:25.763171 4980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:25.767172 4980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:25.767228 4980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:25.767323 4980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:47:25.772593 4980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.129.keytab
--principal=oryx/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:37161
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=33759
--tserver_master_addrs=127.21.236.190:40517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:25.773686 4980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:25.774760 4980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:25.784524 5026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:25.784572 5030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:25.784538 5027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:25.785195 4980 server_base.cc:1034] running on GCE node
I20260811 02:47:25.785601 4980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:25.786290 4980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:25.787462 4980 hybrid_clock.cc:648] HybridClock initialized: now 1786416445787433 us; error 47 us; skew 500 ppm
Aug 11 02:47:25 dist-test-slave-n326 krb5kdc[3917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416445, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:25.790431 4980 init.cc:377] Logged in from keytab as oryx/127.21.236.129@KRBTEST.COM (short username oryx)
I20260811 02:47:25.791329 4980 webserver.cc:460] Webserver started at http://127.21.236.129:33759/ using document root <none> and password file <none>
I20260811 02:47:25.791903 4980 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:25.792021 4980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:25.795959 4980 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:47:25.798110 5043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:25.799441 4980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:47:25.799592 4980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "7ffa036cae64465eaaa6468eb93fbb53"
format_stamp: "Formatted at 2026-08-11 02:47:24 on dist-test-slave-n326"
server_key: "fedb4b70526c5f6973d62ec2edb950b7"
server_key_iv: "1db4633ef9fa6f35f7adf67517fd6963"
server_key_version: "encryptionkey@0"
I20260811 02:47:25.800112 4980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:25.812059 4980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:25.815272 4980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:25.815471 4980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:25.816040 4980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:25.816851 4980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:25.816923 4980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:25.816995 4980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:25.817039 4980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:25.825554 4980 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:37161
I20260811 02:47:25.825627 5250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:37161 every 8 connection(s)
I20260811 02:47:25.826565 4980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:47:25.829293 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 4980
I20260811 02:47:25.829766 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:32905
--local_ip_for_outbound_sockets=127.21.236.130
--tserver_master_addrs=127.21.236.190:40517
--webserver_port=43113
--webserver_interface=127.21.236.130
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:45229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.236.130
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:47:25 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:25 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 02:47:25.842361 5255 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:25.828561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:40135 (local address 127.21.236.190:40517)
0811 02:47:25.828994 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:25.829017 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:25.829265 (+ 248us) server_negotiation.cc:407] Connection header received
0811 02:47:25.830910 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:25.830928 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:25.831361 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:25.831756 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:25.833282 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.834254 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.836214 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.836676 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.836797 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:25.841618 (+ 4821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:40135: BlockingRecv error: recv got EOF from 127.21.236.129:40135 (error 108)
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":134,"threads_started":1}
I20260811 02:47:25.842370 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:25.828634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:40135)
0811 02:47:25.829077 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:25.829130 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:25.830454 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:25.831875 (+ 1421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:25.831885 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:25.832405 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:25.833091 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:25.833116 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.834405 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.834411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:25.836084 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:25.836099 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.836855 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.836862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:25.837105 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:25.837765 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:25.837792 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:25.841354 (+ 3562us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":129,"threads_started":1}
W20260811 02:47:25.842926 5251 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:40517 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.236.190:40517: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Aug 11 02:47:25 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:25 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:25.848733 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:25.843393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:60127)
0811 02:47:25.843607 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:25.843621 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:25.843713 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:25.843980 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:25.843983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:25.844158 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:25.844248 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:25.844253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.845087 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.845091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:25.845642 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:25.845650 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.846111 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.846115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:25.846213 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:25.846742 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:25.846763 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:25.848589 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139}
W20260811 02:47:25.848834 5255 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:25.843480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:60127 (local address 127.21.236.190:40517)
0811 02:47:25.843583 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:25.843587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:25.843667 (+ 80us) server_negotiation.cc:407] Connection header received
0811 02:47:25.843819 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:25.843822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:25.843870 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:25.843967 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:25.844362 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.844972 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.845764 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.845994 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.846102 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:25.848721 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:60127: BlockingRecv error: recv got EOF from 127.21.236.129:60127 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 11 02:47:25 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:25 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:25.854043 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:25.849299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:44807)
0811 02:47:25.849390 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:25.849405 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:25.849479 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:25.849716 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:25.849719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:25.849873 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:25.849939 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:25.849943 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.850640 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.850645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:25.851230 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:25.851239 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.851688 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.851691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:25.851772 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:25.852244 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:25.852264 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:25.853925 (+ 1661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 02:47:25.854174 5255 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:25.849394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:44807 (local address 127.21.236.190:40517)
0811 02:47:25.849505 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:25.849511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:25.849527 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:47:25.849563 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:25.849565 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:25.849613 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:25.849702 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:25.850048 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.850527 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.851344 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:25.851577 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:25.851685 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:25.854041 (+ 2356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:44807: BlockingRecv error: recv got EOF from 127.21.236.129:44807 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 02:47:25.854398 5251 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:47:25.945688 5257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:25.945990 5257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:25.946074 5257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:25.950067 5257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:25.950157 5257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:25.950249 5257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:47:25.955456 5257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.130.keytab
--principal=oryx/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:32905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=43113
--tserver_master_addrs=127.21.236.190:40517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:25.956516 5257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:25.957535 5257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:25.967105 5270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:25.967103 5271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:25.967129 5279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:25.967433 5257 server_base.cc:1034] running on GCE node
I20260811 02:47:25.967849 5257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:25.968447 5257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:25.969655 5257 hybrid_clock.cc:648] HybridClock initialized: now 1786416445969636 us; error 26 us; skew 500 ppm
Aug 11 02:47:25 dist-test-slave-n326 krb5kdc[3917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416445, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:25.973227 5257 init.cc:377] Logged in from keytab as oryx/127.21.236.130@KRBTEST.COM (short username oryx)
I20260811 02:47:25.974313 5257 webserver.cc:460] Webserver started at http://127.21.236.130:43113/ using document root <none> and password file <none>
I20260811 02:47:25.975021 5257 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:25.975186 5257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:25.979683 5257 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260811 02:47:25.982401 5291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:25.983920 5257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:47:25.984066 5257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "6ed7cf0f74d540b0a4b295753ecf1bd8"
format_stamp: "Formatted at 2026-08-11 02:47:25 on dist-test-slave-n326"
server_key: "992aeebf69c879e1efac2fff73706696"
server_key_iv: "a015d92d6b654f788a04f75ee41bc064"
server_key_version: "encryptionkey@0"
I20260811 02:47:25.984704 5257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:26.005633 5257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:26.009119 5257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:26.009339 5257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:26.009979 5257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:26.010845 5257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:26.010901 5257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:26.010943 5257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:26.010984 5257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:26.020916 5257 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:32905
I20260811 02:47:26.020925 5490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:32905 every 8 connection(s)
I20260811 02:47:26.022105 5257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:47:26.027784 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 5257
I20260811 02:47:26.028312 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:46539
--local_ip_for_outbound_sockets=127.21.236.131
--tserver_master_addrs=127.21.236.190:40517
--webserver_port=36887
--webserver_interface=127.21.236.131
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:45229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.236.131
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 02:47:26.034945 5255 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:26.024168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:60705 (local address 127.21.236.190:40517)
0811 02:47:26.024349 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:26.024354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:26.025316 (+ 962us) server_negotiation.cc:407] Connection header received
0811 02:47:26.026627 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:26.026630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:26.026683 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:26.026762 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:26.028185 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.028830 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.030560 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.030942 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.031106 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.034817 (+ 3711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:60705: BlockingRecv error: recv got EOF from 127.21.236.130:60705 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 02:47:26.035612 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:26.024655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:60705)
0811 02:47:26.025155 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:26.025200 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:26.026401 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:26.026931 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:26.026938 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:26.027306 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:26.028015 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:26.028037 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.028967 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.028988 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:26.030427 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:26.030437 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.031163 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.031167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:26.031309 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.031989 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:26.032015 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:26.034694 (+ 2679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":126,"threads_started":1}
W20260811 02:47:26.036141 5491 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:40517 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.236.190:40517: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:26.041604 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:26.036628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:38669)
0811 02:47:26.036738 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:26.036754 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:26.036835 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:26.037107 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:26.037110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:26.037263 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:26.037349 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:26.037354 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.038064 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.038067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:26.038607 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:26.038615 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.039091 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.039096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:26.039195 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.039699 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:26.039710 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:26.041469 (+ 1759us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 02:47:26.041729 5255 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:26.036705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:38669 (local address 127.21.236.190:40517)
0811 02:47:26.036818 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:26.036822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:26.036835 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:47:26.036950 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:26.036953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:26.037003 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:26.037092 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:26.037461 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.037951 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.038727 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.038951 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.039059 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.041596 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:38669: BlockingRecv error: recv got EOF from 127.21.236.130:38669 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:26.047574 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:26.042352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:38035)
0811 02:47:26.042432 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:26.042443 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:26.042500 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:26.042826 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:26.042829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:26.042970 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:26.043046 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:26.043051 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.043786 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.043792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:26.044303 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:26.044311 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.044764 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.044768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:26.044844 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.045296 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:26.045307 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:26.047442 (+ 2135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 02:47:26.047732 5255 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:26.042521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:38035 (local address 127.21.236.190:40517)
0811 02:47:26.042650 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:26.042654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:26.042664 (+ 10us) server_negotiation.cc:407] Connection header received
0811 02:47:26.042692 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:26.042694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:26.042726 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:26.042811 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:26.043205 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.043681 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.044419 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.044649 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.044762 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.047578 (+ 2816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:38035: BlockingRecv error: recv got EOF from 127.21.236.130:38035 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 02:47:26.047899 5491 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:47:26.144197 5495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:26.144459 5495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:26.144502 5495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:26.148654 5495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:26.148717 5495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:26.148804 5495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:47:26.153971 5495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/krb5kdc/oryx_127.21.236.131.keytab
--principal=oryx/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:46539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=36887
--tserver_master_addrs=127.21.236.190:40517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:26.155112 5495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:26.156191 5495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:26.166255 5509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:26.166277 5503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:26.166271 5505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:26.166518 5495 server_base.cc:1034] running on GCE node
I20260811 02:47:26.167034 5495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:26.167738 5495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:26.168923 5495 hybrid_clock.cc:648] HybridClock initialized: now 1786416446168908 us; error 25 us; skew 500 ppm
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416446, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:26.171720 5495 init.cc:377] Logged in from keytab as oryx/127.21.236.131@KRBTEST.COM (short username oryx)
I20260811 02:47:26.172572 5495 webserver.cc:460] Webserver started at http://127.21.236.131:36887/ using document root <none> and password file <none>
I20260811 02:47:26.173142 5495 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:26.173214 5495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:26.177413 5495 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:47:26.179442 5518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:26.180670 5495 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:47:26.180815 5495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "041aed63510e473a83b151b43eaebe26"
format_stamp: "Formatted at 2026-08-11 02:47:25 on dist-test-slave-n326"
server_key: "880903d6d863ffc5b5269fb4e7361cb8"
server_key_iv: "9b0990f95a3b4d6f227dd92caa6dacab"
server_key_version: "encryptionkey@0"
I20260811 02:47:26.181470 5495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:26.222152 5495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:26.225414 5495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:26.225638 5495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:26.226256 5495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:26.227208 5495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:26.227258 5495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:26.227325 5495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:26.227361 5495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:26.236841 5495 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:46539
I20260811 02:47:26.236873 5727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:46539 every 8 connection(s)
I20260811 02:47:26.237920 5495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:47:26.247020 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 5495
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 02:47:26.249799 5255 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:26.239878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:59329 (local address 127.21.236.190:40517)
0811 02:47:26.240043 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:26.240046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:26.241026 (+ 980us) server_negotiation.cc:407] Connection header received
0811 02:47:26.242425 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:26.242429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:26.242489 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:26.242596 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:26.243847 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.244391 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.245534 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.245761 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.245874 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.249671 (+ 3797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:59329: BlockingRecv error: recv got EOF from 127.21.236.131:59329 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 02:47:26.250434 5732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:26.240275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:59329)
0811 02:47:26.240807 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:26.240913 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:26.242212 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:26.242781 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:26.242787 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:26.243204 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:26.243693 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:26.243706 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.244524 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.244527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:26.245405 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:26.245415 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.245915 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.245919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:26.246088 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.246659 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:26.246675 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:26.249546 (+ 2871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":112,"threads_started":1}
W20260811 02:47:26.251118 5728 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:40517 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.236.190:40517: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:26.256991 5732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:26.251556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:52095)
0811 02:47:26.251777 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:26.251810 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:26.251896 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:26.252128 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:26.252131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:26.252291 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:26.252415 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:26.252422 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.253120 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.253123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:26.253842 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:26.253856 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.254482 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.254485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:26.254566 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.254982 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:26.254993 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:26.256854 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162}
W20260811 02:47:26.257084 5255 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:26.251674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:52095 (local address 127.21.236.190:40517)
0811 02:47:26.251779 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:26.251783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:26.251856 (+ 73us) server_negotiation.cc:407] Connection header received
0811 02:47:26.251984 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:26.251986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:26.252015 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:26.252087 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:26.252534 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.253002 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.253965 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.254359 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.254469 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.256963 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:52095: BlockingRecv error: recv got EOF from 127.21.236.131:52095 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 02:47:26.257634 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:26.248098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.21.236.190:40517)
0811 02:47:26.248406 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:26.248413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:26.248431 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:47:26.248479 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:26.248482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:26.248540 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:26.248632 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:26.249568 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.250202 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.251395 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.251736 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.251839 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.253297 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:26.253303 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:26.253316 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:26.253340 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:26.255218 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:26.255722 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:26.255735 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:26.255740 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:26.255883 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:26.256159 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:26.256163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:26.256165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:26.256539 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:26.256756 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:26.257006 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 02:47:26.257465 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":113,"threads_started":1}
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:26.263027 5732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:26.257573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:48997)
0811 02:47:26.258532 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:26.258543 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:26.258621 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:26.258831 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:26.258833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:26.258951 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:26.259013 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:26.259017 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.259903 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.259907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:26.260572 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:26.260581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.260953 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.260955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:26.261007 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.261400 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:26.261411 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:26.262945 (+ 1534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":902}
W20260811 02:47:26.263248 5728 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:47:26.263430 5255 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:26.257693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:48997 (local address 127.21.236.190:40517)
0811 02:47:26.257831 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:26.257835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:26.258613 (+ 778us) server_negotiation.cc:407] Connection header received
0811 02:47:26.258694 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:26.258697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:26.258738 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:26.258809 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:26.259138 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.259791 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.260670 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.260863 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.260942 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.263057 (+ 2115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:48997: BlockingRecv error: recv got EOF from 127.21.236.131:48997 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:26 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:26.860921 5742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:26.855052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:59483)
0811 02:47:26.855368 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:26.855382 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:26.855481 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:26.855741 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:26.855744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:26.855922 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:26.856024 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:26.856029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.856748 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.856750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:26.857425 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:26.857435 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.857938 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.857942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:26.858024 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.858634 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:26.858646 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:26.860768 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":89,"threads_started":1}
W20260811 02:47:26.861310 5743 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:26.855165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:59483 (local address 127.21.236.190:40517)
0811 02:47:26.855410 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:26.855414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:26.855432 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:47:26.855577 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:26.855579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:26.855627 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:26.855706 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:26.856129 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.856654 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.857567 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:26.857814 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:26.857931 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:26.860984 (+ 3053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:59483: BlockingRecv error: recv got EOF from 127.21.236.129:59483 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":62,"threads_started":1}
Aug 11 02:47:27 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:27 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:27.054543 5744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:27.048671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:49701)
0811 02:47:27.049076 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:27.049091 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:27.049232 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:27.049512 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:27.049515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:27.049734 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:27.049831 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:27.049837 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:27.050615 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:27.050621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:27.051232 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:27.051241 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:27.051736 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:27.051740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:27.051829 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:27.052407 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:27.052419 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:27.054406 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":97,"threads_started":1}
W20260811 02:47:27.054852 5743 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:27.048816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:49701 (local address 127.21.236.190:40517)
0811 02:47:27.049023 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:27.049028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:27.049169 (+ 141us) server_negotiation.cc:407] Connection header received
0811 02:47:27.049338 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:27.049342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:27.049396 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:27.049486 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:27.049954 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:27.050492 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:27.051354 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:27.051612 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:27.051729 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:27.054562 (+ 2833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:49701: BlockingRecv error: recv got EOF from 127.21.236.130:49701 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 11 02:47:27 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:27 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:27.269824 5745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:27.263954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:41399)
0811 02:47:27.264258 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:27.264271 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:27.264403 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:27.264690 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:27.264693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:27.264901 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:27.264994 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:27.265000 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:27.265774 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:27.265779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:27.266369 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:27.266378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:27.266884 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:27.266888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:27.266984 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:27.267602 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:27.267614 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:27.269651 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":94,"threads_started":1}
W20260811 02:47:27.269933 5743 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:27.264013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:41399 (local address 127.21.236.190:40517)
0811 02:47:27.264194 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:27.264199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:27.264345 (+ 146us) server_negotiation.cc:407] Connection header received
0811 02:47:27.264515 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:27.264520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:27.264578 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:27.264664 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:27.265120 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:27.265658 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:27.266493 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:27.266731 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:27.266867 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:27.269772 (+ 2905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:41399: BlockingRecv error: recv got EOF from 127.21.236.131:41399 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 02:47:27 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:27 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:27.867534 5747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:27.861808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:33641)
0811 02:47:27.862138 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:27.862166 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:27.862268 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:27.862571 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:27.862573 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:27.862777 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:27.862886 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:27.862892 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:27.863785 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:27.863788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:27.864388 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:27.864397 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:27.864811 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:27.864814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:27.864883 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:27.865486 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:27.865498 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:27.867396 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":86,"threads_started":1}
W20260811 02:47:27.867673 5748 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:27.861881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:33641 (local address 127.21.236.190:40517)
0811 02:47:27.862240 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:27.862243 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:27.862257 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:47:27.862366 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:27.862368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:27.862416 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:27.862502 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:27.863032 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:27.863656 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:27.864493 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:27.864712 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:27.864786 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:27.867518 (+ 2732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:33641: BlockingRecv error: recv got EOF from 127.21.236.129:33641 (error 108)
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":149,"threads_started":1}
Aug 11 02:47:28 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:28 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:28.061292 5749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:28.055394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:52071)
0811 02:47:28.055741 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:28.055760 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:28.055886 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:28.056246 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:28.056249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:28.056511 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:28.056651 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:28.056664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:28.057587 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:28.057590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:28.058156 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:28.058164 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:28.058576 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:28.058579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:28.058639 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:28.059242 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:28.059257 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:28.061150 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":96,"threads_started":1}
W20260811 02:47:28.061460 5748 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:28.055565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:52071 (local address 127.21.236.190:40517)
0811 02:47:28.055736 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:28.055740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:28.055961 (+ 221us) server_negotiation.cc:407] Connection header received
0811 02:47:28.056016 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:28.056021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:28.056111 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:28.056182 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:28.056792 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:28.057478 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:28.058264 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:28.058474 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:28.058554 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:28.061290 (+ 2736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:52071: BlockingRecv error: recv got EOF from 127.21.236.130:52071 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 02:47:28 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:28 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:28.276705 5751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:28.270758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:49817)
0811 02:47:28.271116 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:28.271132 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:28.271281 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:28.271610 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:28.271613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:28.271826 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:28.271932 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:28.271939 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:28.272760 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:28.272764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:28.273328 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:28.273337 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:28.273858 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:28.273862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:28.273960 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:28.274448 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:28.274460 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:28.276556 (+ 2096us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":129,"threads_started":1}
W20260811 02:47:28.276832 5748 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:28.270931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:49817 (local address 127.21.236.190:40517)
0811 02:47:28.271116 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:28.271122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:28.271217 (+ 95us) server_negotiation.cc:407] Connection header received
0811 02:47:28.271392 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:28.271395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:28.271488 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:28.271584 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:28.272057 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:28.272639 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:28.273458 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:28.273715 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:28.273823 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:28.276710 (+ 2887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:49817: BlockingRecv error: recv got EOF from 127.21.236.131:49817 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 02:47:28 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:28 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:28.874076 5753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:28.868417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:35167)
0811 02:47:28.868716 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:28.868730 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:28.868822 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:28.869095 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:28.869098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:28.869280 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:28.869376 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:28.869382 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:28.870246 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:28.870248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:28.870869 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:28.870878 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:28.871445 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:28.871449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:28.871527 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:28.872044 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:28.872056 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:28.873958 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":77,"threads_started":1}
W20260811 02:47:28.874343 5755 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:28.868484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:35167 (local address 127.21.236.190:40517)
0811 02:47:28.868758 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:28.868762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:28.868777 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:47:28.868908 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:28.868910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:28.868955 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:28.869036 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:28.869510 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:28.870123 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:28.870985 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:28.871308 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:28.871392 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:28.874210 (+ 2818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:35167: BlockingRecv error: recv got EOF from 127.21.236.129:35167 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":124,"threads_started":1}
Aug 11 02:47:29 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:29 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:29.067822 5756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:29.062141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:60967)
0811 02:47:29.062440 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:29.062455 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:29.062565 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:29.062849 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:29.062852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:29.063095 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:29.063205 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:29.063211 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:29.063949 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:29.063953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:29.064566 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:29.064574 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:29.064990 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:29.064993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:29.065061 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:29.065556 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:29.065568 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:29.067696 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":70,"threads_started":1}
W20260811 02:47:29.068003 5755 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:29.062301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:60967 (local address 127.21.236.190:40517)
0811 02:47:29.062460 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:29.062463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:29.062533 (+ 70us) server_negotiation.cc:407] Connection header received
0811 02:47:29.062650 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:29.062652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:29.062698 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:29.062768 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:29.063315 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:29.063845 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:29.064672 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:29.064880 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:29.064956 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:29.067833 (+ 2877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:60967: BlockingRecv error: recv got EOF from 127.21.236.130:60967 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 02:47:29 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:29 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:29.283450 5757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:29.277670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:46399)
0811 02:47:29.277987 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:29.278001 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:29.278120 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:29.278366 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:29.278368 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:29.278559 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:29.278657 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:29.278663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:29.279493 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:29.279496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:29.280051 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:29.280060 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:29.280494 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:29.280496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:29.280560 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:29.281075 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:29.281088 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:29.283272 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":132,"threads_started":1}
W20260811 02:47:29.283509 5755 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:29.277753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:46399 (local address 127.21.236.190:40517)
0811 02:47:29.277918 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:29.277923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:29.278115 (+ 192us) server_negotiation.cc:407] Connection header received
0811 02:47:29.278194 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:29.278196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:29.278252 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:29.278332 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:29.278779 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:29.279396 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:29.280161 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:29.280394 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:29.280480 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:29.283397 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:46399: BlockingRecv error: recv got EOF from 127.21.236.131:46399 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 02:47:29 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:29 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:29.885773 5758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:29.874942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:33135)
0811 02:47:29.875330 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:29.875345 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:29.875458 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:29.875681 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:29.875684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:29.875901 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:29.876007 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:29.876013 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:29.876836 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:29.876839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:29.877479 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:29.877488 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:29.877960 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:29.877963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:29.878034 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:29.883248 (+ 5214us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:29.883262 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:29.885592 (+ 2330us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":164,"threads_started":1}
W20260811 02:47:29.885857 5762 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:29.875011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:33135 (local address 127.21.236.190:40517)
0811 02:47:29.875333 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:29.875336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:29.875405 (+ 69us) server_negotiation.cc:407] Connection header received
0811 02:47:29.875536 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:29.875538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:29.875582 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:29.875661 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:29.876113 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:29.876720 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:29.877612 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:29.877845 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:29.877935 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:29.885727 (+ 7792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:33135: BlockingRecv error: recv got EOF from 127.21.236.129:33135 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":101,"threads_started":1}
Aug 11 02:47:30 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:30 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:30.074368 5763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:30.068684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:55939)
0811 02:47:30.068961 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:30.068974 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:30.069079 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:30.069301 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:30.069303 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:30.069484 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:30.069581 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:30.069587 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:30.070360 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:30.070362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:30.070917 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:30.070925 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:30.071539 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:30.071543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:30.071633 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:30.072171 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:30.072183 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:30.074160 (+ 1977us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":107,"threads_started":1}
W20260811 02:47:30.074474 5762 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:30.068756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:55939 (local address 127.21.236.190:40517)
0811 02:47:30.068908 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:30.068911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:30.069052 (+ 141us) server_negotiation.cc:407] Connection header received
0811 02:47:30.069148 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:30.069150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:30.069193 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:30.069263 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:30.069682 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:30.070244 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:30.071007 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:30.071400 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:30.071524 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:30.074349 (+ 2825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:55939: BlockingRecv error: recv got EOF from 127.21.236.130:55939 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 02:47:30 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:30 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:30.290104 5765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:30.284311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:42223)
0811 02:47:30.284639 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:30.284652 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:30.284787 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:30.285146 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:30.285150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:30.285396 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:30.285504 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:30.285511 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:30.286275 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:30.286279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:30.286912 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:30.286921 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:30.287449 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:30.287453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:30.287543 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:30.288026 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:30.288038 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:30.289976 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":95,"threads_started":1}
W20260811 02:47:30.290241 5762 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:30.284422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:42223 (local address 127.21.236.190:40517)
0811 02:47:30.284569 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:30.284574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:30.284725 (+ 151us) server_negotiation.cc:407] Connection header received
0811 02:47:30.284901 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:30.284903 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:30.284952 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:30.285105 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:30.285630 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:30.286163 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:30.287040 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:30.287331 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:30.287444 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:30.290118 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:42223: BlockingRecv error: recv got EOF from 127.21.236.131:42223 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 02:47:30 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:30 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:30.893231 5767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:30.886734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:51911)
0811 02:47:30.887170 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:30.887189 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:30.887312 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:30.887596 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:30.887600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:30.887868 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:30.888058 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:30.888069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:30.888922 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:30.888926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:30.889658 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:30.889670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:30.890166 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:30.890170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:30.890260 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:30.890871 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:30.890882 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:30.893042 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":117,"threads_started":1}
W20260811 02:47:30.893313 5768 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:30.886789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:51911 (local address 127.21.236.190:40517)
0811 02:47:30.887028 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:30.887031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:30.887262 (+ 231us) server_negotiation.cc:407] Connection header received
0811 02:47:30.887409 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:30.887411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:30.887458 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:30.887536 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:30.888182 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:30.888786 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:30.889786 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:30.890022 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:30.890145 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:30.893179 (+ 3034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:51911: BlockingRecv error: recv got EOF from 127.21.236.129:51911 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":88,"threads_started":1}
Aug 11 02:47:31 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:31 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:31.081328 5770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:31.075435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:48451)
0811 02:47:31.075758 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:31.075772 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:31.075867 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:31.076143 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:31.076145 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:31.076385 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:31.076486 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:31.076492 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:31.077350 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:31.077352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:31.077909 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:31.077917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:31.078395 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:31.078397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:31.078457 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:31.078988 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:31.079000 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:31.081136 (+ 2136us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":146,"threads_started":1}
W20260811 02:47:31.081414 5768 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:31.075504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:48451 (local address 127.21.236.190:40517)
0811 02:47:31.075673 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:31.075678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:31.075824 (+ 146us) server_negotiation.cc:407] Connection header received
0811 02:47:31.075935 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:31.075938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:31.075983 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:31.076121 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:31.076610 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:31.077241 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:31.078038 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:31.078289 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:31.078389 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:31.081276 (+ 2887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:48451: BlockingRecv error: recv got EOF from 127.21.236.130:48451 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 02:47:31 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:31 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:31.297399 5772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:31.291178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:59085)
0811 02:47:31.291509 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:31.291553 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:31.291660 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:31.291916 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:31.291918 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:31.292124 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:31.292223 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:31.292229 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:31.292969 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:31.292972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:31.293631 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:31.293640 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:31.294090 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:31.294093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:31.294161 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:31.294685 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:31.294697 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:31.297224 (+ 2527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":131,"threads_started":1}
W20260811 02:47:31.297458 5768 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:31.291234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:59085 (local address 127.21.236.190:40517)
0811 02:47:31.291402 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:31.291407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:31.291661 (+ 254us) server_negotiation.cc:407] Connection header received
0811 02:47:31.291754 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:31.291756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:31.291808 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:31.291871 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:31.292334 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:31.292868 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:31.293753 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:31.293977 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:31.294061 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:31.297348 (+ 3287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:59085: BlockingRecv error: recv got EOF from 127.21.236.131:59085 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 02:47:31 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:31 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:31.900246 5773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:31.894126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:50771)
0811 02:47:31.894441 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:31.894462 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:31.894552 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:31.894827 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:31.894829 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:31.895014 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:31.895173 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:31.895179 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:31.896054 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:31.896056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:31.896600 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:31.896607 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:31.897173 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:31.897175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:31.897283 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:31.897960 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:31.897975 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:31.900055 (+ 2080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":130,"threads_started":1}
W20260811 02:47:31.900308 5775 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:31.894289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:50771 (local address 127.21.236.190:40517)
0811 02:47:31.894558 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:31.894562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:31.894608 (+ 46us) server_negotiation.cc:407] Connection header received
0811 02:47:31.894661 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:31.894664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:31.894711 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:31.894790 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:31.895312 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:31.895927 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:31.896709 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:31.897044 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:31.897154 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:31.900182 (+ 3028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:50771: BlockingRecv error: recv got EOF from 127.21.236.129:50771 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":106,"threads_started":1}
Aug 11 02:47:32 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:32 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:32.088403 5776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:32.082305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:57183)
0811 02:47:32.082617 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:32.082634 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:32.082788 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:32.083132 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:32.083135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:32.083355 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:32.083456 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:32.083462 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:32.084261 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:32.084267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:32.084913 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:32.084923 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:32.085381 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:32.085385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:32.085474 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:32.086142 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:32.086158 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:32.088283 (+ 2125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":93,"threads_started":1}
W20260811 02:47:32.088548 5775 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:32.082447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:57183 (local address 127.21.236.190:40517)
0811 02:47:32.082618 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:32.082624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:32.082720 (+ 96us) server_negotiation.cc:407] Connection header received
0811 02:47:32.082913 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:32.082917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:32.082971 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:32.083061 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:32.083607 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:32.084141 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:32.085030 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:32.085260 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:32.085374 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:32.088428 (+ 3054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:57183: BlockingRecv error: recv got EOF from 127.21.236.130:57183 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 02:47:32 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:32 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:32.304210 5777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:32.298380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:38663)
0811 02:47:32.298655 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:32.298668 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:32.298801 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:32.299132 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:32.299135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:32.299346 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:32.299436 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:32.299442 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:32.300221 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:32.300226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:32.300787 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:32.300795 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:32.301300 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:32.301303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:32.301421 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:32.301909 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:32.301921 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:32.304093 (+ 2172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":97,"threads_started":1}
W20260811 02:47:32.304363 5775 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:32.298438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:38663 (local address 127.21.236.190:40517)
0811 02:47:32.298587 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:32.298592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:32.298747 (+ 155us) server_negotiation.cc:407] Connection header received
0811 02:47:32.298902 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:32.298905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:32.298956 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:32.299050 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:32.299569 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:32.300109 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:32.300911 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:32.301179 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:32.301287 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:32.304242 (+ 2955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:38663: BlockingRecv error: recv got EOF from 127.21.236.131:38663 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 02:47:32 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:32 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:32.906980 5779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:32.901205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:47645)
0811 02:47:32.901549 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:32.901563 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:32.901669 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:32.901910 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:32.901912 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:32.902106 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:32.902202 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:32.902207 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:32.903142 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:32.903146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:32.903795 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:32.903804 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:32.904368 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:32.904371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:32.904443 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:32.905020 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:32.905033 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:32.906860 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":102,"threads_started":1}
W20260811 02:47:32.907202 5778 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:32.901245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:47645 (local address 127.21.236.190:40517)
0811 02:47:32.901558 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:32.901563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:32.901636 (+ 73us) server_negotiation.cc:407] Connection header received
0811 02:47:32.901745 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:32.901747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:32.901793 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:32.901870 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:32.902301 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:32.902987 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:32.903924 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:32.904255 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:32.904341 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:32.907013 (+ 2672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:47645: BlockingRecv error: recv got EOF from 127.21.236.129:47645 (error 108)
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":105,"threads_started":1}
Aug 11 02:47:33 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:33 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:33.094718 5780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:33.089230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:42341)
0811 02:47:33.089634 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:33.089649 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:33.089768 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:33.090052 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:33.090055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:33.090255 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:33.090355 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:33.090361 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:33.091167 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:33.091170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:33.091759 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:33.091768 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:33.092189 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:33.092191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:33.092255 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:33.092752 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:33.092764 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:33.094599 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":98,"threads_started":1}
W20260811 02:47:33.094859 5778 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:33.089315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:42341 (local address 127.21.236.190:40517)
0811 02:47:33.089507 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:33.089511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:33.089736 (+ 225us) server_negotiation.cc:407] Connection header received
0811 02:47:33.089852 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:33.089855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:33.089902 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:33.089972 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:33.090462 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:33.090984 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:33.091864 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:33.092087 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:33.092163 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:33.094738 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:42341: BlockingRecv error: recv got EOF from 127.21.236.130:42341 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 02:47:33 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:33 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:33.310843 5781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:33.305167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:50827)
0811 02:47:33.305551 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:33.305566 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:33.305679 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:33.305987 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:33.305990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:33.306189 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:33.306284 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:33.306289 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:33.307049 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:33.307051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:33.307671 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:33.307680 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:33.308091 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:33.308094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:33.308157 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:33.308766 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:33.308778 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:33.310729 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":127,"threads_started":1}
W20260811 02:47:33.311012 5778 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:33.305245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:50827 (local address 127.21.236.190:40517)
0811 02:47:33.305410 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:33.305416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:33.305649 (+ 233us) server_negotiation.cc:407] Connection header received
0811 02:47:33.305753 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:33.305755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:33.305806 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:33.305874 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:33.306395 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:33.306945 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:33.307773 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:33.307989 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:33.308065 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:33.310854 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:50827: BlockingRecv error: recv got EOF from 127.21.236.131:50827 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 02:47:33 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:33 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:33.913419 5783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:33.907848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:45297)
0811 02:47:33.908162 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:33.908176 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:33.908266 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:33.908506 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:33.908508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:33.908692 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:33.908791 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:33.908796 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:33.909620 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:33.909623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:33.910187 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:33.910195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:33.910720 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:33.910723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:33.910786 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:33.911363 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:33.911375 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:33.913299 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":116,"threads_started":1}
W20260811 02:47:33.913653 5787 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:33.908022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:45297 (local address 127.21.236.190:40517)
0811 02:47:33.908251 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:33.908255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:33.908269 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:47:33.908338 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:33.908340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:33.908385 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:33.908465 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:33.908900 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:33.909514 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:33.910321 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:33.910611 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:33.910706 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:33.913496 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:45297: BlockingRecv error: recv got EOF from 127.21.236.129:45297 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":68,"threads_started":1}
Aug 11 02:47:34 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:34 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:34.101792 5788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:34.095784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:51085)
0811 02:47:34.096072 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:34.096086 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:34.096188 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:34.096463 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:34.096466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:34.096654 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:34.096758 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:34.096763 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:34.097657 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:34.097659 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:34.098218 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:34.098226 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:34.098728 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:34.098731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:34.098799 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:34.099475 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:34.099489 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:34.101668 (+ 2179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":114,"threads_started":1}
W20260811 02:47:34.101971 5787 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:34.095851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:51085 (local address 127.21.236.190:40517)
0811 02:47:34.096024 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:34.096027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:34.096157 (+ 130us) server_negotiation.cc:407] Connection header received
0811 02:47:34.096272 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:34.096274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:34.096330 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:34.096400 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:34.096862 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:34.097551 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:34.098336 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:34.098579 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:34.098669 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:34.101810 (+ 3141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:51085: BlockingRecv error: recv got EOF from 127.21.236.130:51085 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 02:47:34 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:34 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:34.317643 5789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:34.311746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:60041)
0811 02:47:34.312038 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:34.312052 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:34.312162 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:34.312385 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:34.312386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:34.312581 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:34.312680 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:34.312686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:34.313444 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:34.313446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:34.314033 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:34.314042 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:34.314489 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:34.314492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:34.314558 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:34.315257 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:34.315275 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:34.317523 (+ 2248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":73,"threads_started":1}
W20260811 02:47:34.317848 5787 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:34.311911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:60041 (local address 127.21.236.190:40517)
0811 02:47:34.312074 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:34.312078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:34.312109 (+ 31us) server_negotiation.cc:407] Connection header received
0811 02:47:34.312250 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:34.312252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:34.312301 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:34.312373 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:34.312781 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:34.313307 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:34.314155 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:34.314370 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:34.314463 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:34.317693 (+ 3230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:60041: BlockingRecv error: recv got EOF from 127.21.236.131:60041 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 02:47:34 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:34 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:34.927304 5790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:34.914227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:51829)
0811 02:47:34.914560 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:34.914574 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:34.914668 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:34.915005 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:34.915007 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:34.915219 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:34.915322 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:34.915328 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:34.916133 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:34.916136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:34.916726 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:34.916735 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:34.917293 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:34.917296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:34.917365 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:34.925132 (+ 7767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:34.925145 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:34.927147 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":137,"threads_started":1}
W20260811 02:47:34.927459 5791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:34.914517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:51829 (local address 127.21.236.190:40517)
0811 02:47:34.914754 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:34.914758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:34.914770 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:47:34.914805 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:34.914807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:34.914847 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:34.914956 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:34.915464 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:34.916015 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:34.916910 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:34.917165 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:34.917272 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:34.927293 (+ 10021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:51829: BlockingRecv error: recv got EOF from 127.21.236.129:51829 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":91,"threads_started":1}
Aug 11 02:47:35 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:35 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:35.108601 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:35.102695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:40371)
0811 02:47:35.103056 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:35.103091 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:35.103196 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:35.103444 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:35.103447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:35.103651 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:35.103752 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:35.103757 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:35.104652 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:35.104654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:35.105204 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:35.105212 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:35.105657 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:35.105660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:35.105722 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:35.106243 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:35.106254 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:35.108443 (+ 2189us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":105,"threads_started":1}
W20260811 02:47:35.108716 5791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:35.102756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:40371 (local address 127.21.236.190:40517)
0811 02:47:35.102916 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:35.102921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:35.103171 (+ 250us) server_negotiation.cc:407] Connection header received
0811 02:47:35.103275 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:35.103277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:35.103324 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:35.103398 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:35.103856 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:35.104550 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:35.105321 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:35.105552 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:35.105637 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:35.108601 (+ 2964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:40371: BlockingRecv error: recv got EOF from 127.21.236.130:40371 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 02:47:35 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:35 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:35.324317 5793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:35.318576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:40245)
0811 02:47:35.318907 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:35.318920 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:35.319057 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:35.319400 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:35.319403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:35.319593 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:35.319696 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:35.319702 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:35.320448 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:35.320452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:35.321020 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:35.321028 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:35.321586 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:35.321589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:35.321652 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:35.322146 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:35.322158 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:35.324179 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":125,"threads_started":1}
W20260811 02:47:35.324447 5791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:35.318606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:40245 (local address 127.21.236.190:40517)
0811 02:47:35.318789 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:35.318795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:35.319003 (+ 208us) server_negotiation.cc:407] Connection header received
0811 02:47:35.319196 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:35.319199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:35.319257 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:35.319357 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:35.319806 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:35.320338 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:35.321163 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:35.321471 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:35.321574 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:35.324324 (+ 2750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:40245: BlockingRecv error: recv got EOF from 127.21.236.131:40245 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 02:47:35 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:35 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:35.933703 5794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:35.928139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:54943)
0811 02:47:35.928473 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:35.928488 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:35.928581 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:35.928933 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:35.928935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:35.929137 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:35.929236 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:35.929241 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:35.930022 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:35.930025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:35.930601 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:35.930609 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:35.931059 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:35.931061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:35.931215 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:35.931760 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:35.931771 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:35.933580 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":87,"threads_started":1}
W20260811 02:47:35.933959 5795 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:35.928273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:54943 (local address 127.21.236.190:40517)
0811 02:47:35.928545 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:35.928548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:35.928563 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:47:35.928717 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:35.928719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:35.928769 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:35.928852 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:35.929358 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:35.929913 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:35.930718 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:35.930954 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:35.931041 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:35.933763 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:54943: BlockingRecv error: recv got EOF from 127.21.236.129:54943 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":105,"threads_started":1}
Aug 11 02:47:36 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:36 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:36.115720 5797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:36.109551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:48199)
0811 02:47:36.109884 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:36.109904 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:36.110039 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:36.110371 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:36.110374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:36.110660 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:36.110797 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:36.110806 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:36.111874 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:36.111878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:36.112436 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:36.112445 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:36.112964 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:36.112967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:36.113042 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:36.113613 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:36.113634 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:36.115598 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":109,"threads_started":1}
W20260811 02:47:36.115911 5795 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:36.109679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:48199 (local address 127.21.236.190:40517)
0811 02:47:36.109845 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:36.109852 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:36.110061 (+ 209us) server_negotiation.cc:407] Connection header received
0811 02:47:36.110158 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:36.110162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:36.110228 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:36.110338 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:36.110981 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:36.111733 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:36.112581 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:36.112806 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:36.112907 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:36.115761 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:48199: BlockingRecv error: recv got EOF from 127.21.236.130:48199 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 02:47:36 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:36 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:36.330709 5799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:36.325182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:36231)
0811 02:47:36.325507 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:36.325524 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:36.325639 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:36.325906 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:36.325909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:36.326135 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:36.326228 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:36.326234 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:36.327007 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:36.327013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:36.327626 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:36.327635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:36.328115 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:36.328119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:36.328208 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:36.328714 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:36.328725 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:36.330550 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":109,"threads_started":1}
W20260811 02:47:36.330823 5795 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:36.325353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:36231 (local address 127.21.236.190:40517)
0811 02:47:36.325507 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:36.325513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:36.325617 (+ 104us) server_negotiation.cc:407] Connection header received
0811 02:47:36.325725 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:36.325729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:36.325784 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:36.325877 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:36.326374 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:36.326896 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:36.327753 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:36.327990 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:36.328104 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:36.330707 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:36231: BlockingRecv error: recv got EOF from 127.21.236.131:36231 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 02:47:36 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:36 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:36.940452 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:36.934725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:38233)
0811 02:47:36.935109 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:36.935128 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:36.935252 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:36.935527 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:36.935530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:36.935736 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:36.935837 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:36.935843 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:36.936594 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:36.936597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:36.937171 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:36.937179 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:36.937640 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:36.937642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:36.937706 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:36.938254 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:36.938266 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:36.940333 (+ 2067us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":90,"threads_started":1}
W20260811 02:47:36.940860 5803 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:36.934775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:38233 (local address 127.21.236.190:40517)
0811 02:47:36.935059 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:36.935065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:36.935221 (+ 156us) server_negotiation.cc:407] Connection header received
0811 02:47:36.935321 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:36.935324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:36.935370 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:36.935448 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:36.935959 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:36.936494 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:36.937282 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:36.937514 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:36.937622 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:36.940703 (+ 3081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:38233: BlockingRecv error: recv got EOF from 127.21.236.129:38233 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":89,"threads_started":1}
Aug 11 02:47:37 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:37 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:37.122167 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:37.116583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:35423)
0811 02:47:37.116873 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:37.116888 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:37.117002 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:37.117247 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:37.117250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:37.117485 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:37.117588 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:37.117594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:37.118407 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:37.118410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:37.118978 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:37.118987 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:37.119506 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:37.119509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:37.119574 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:37.120101 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:37.120113 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:37.122040 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":110,"threads_started":1}
W20260811 02:47:37.122313 5803 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:37.116703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:35423 (local address 127.21.236.190:40517)
0811 02:47:37.116854 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:37.116858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:37.116987 (+ 129us) server_negotiation.cc:407] Connection header received
0811 02:47:37.117081 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:37.117083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:37.117133 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:37.117201 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:37.117715 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:37.118251 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:37.119154 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:37.119393 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:37.119495 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:37.122191 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:35423: BlockingRecv error: recv got EOF from 127.21.236.130:35423 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 02:47:37 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:37 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:37.337476 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:37.331612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:57947)
0811 02:47:37.331950 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:37.331963 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:37.332106 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:37.332391 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:37.332395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:37.332623 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:37.332721 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:37.332727 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:37.333623 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:37.333630 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:37.334221 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:37.334230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:37.334721 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:37.334725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:37.334813 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:37.335353 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:37.335365 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:37.337319 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":122,"threads_started":1}
W20260811 02:47:37.337683 5803 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:37.331677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:57947 (local address 127.21.236.190:40517)
0811 02:47:37.331866 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:37.331871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:37.332045 (+ 174us) server_negotiation.cc:407] Connection header received
0811 02:47:37.332212 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:37.332216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:37.332273 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:37.332373 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:37.332848 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:37.333407 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:37.334349 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:37.334596 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:37.334713 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:37.337486 (+ 2773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:57947: BlockingRecv error: recv got EOF from 127.21.236.131:57947 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 02:47:37 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:37 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:37.946992 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:37.941295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:50365)
0811 02:47:37.941629 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:37.941643 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:37.941742 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:37.941995 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:37.941997 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:37.942191 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:37.942291 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:37.942296 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:37.943105 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:37.943108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:37.943678 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:37.943687 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:37.944105 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:37.944107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:37.944172 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:37.944778 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:37.944790 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:37.946872 (+ 2082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":97,"threads_started":1}
W20260811 02:47:37.947181 5816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:37.941350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:50365 (local address 127.21.236.190:40517)
0811 02:47:37.941675 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:37.941679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:37.941692 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:47:37.941829 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:37.941831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:37.941875 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:37.941951 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:37.942402 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:37.942949 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:37.943782 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:37.944004 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:37.944082 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:37.947009 (+ 2927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:50365: BlockingRecv error: recv got EOF from 127.21.236.129:50365 (error 108)
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":133,"threads_started":1}
Aug 11 02:47:38 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:38 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:38.128564 5817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:38.123231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:52151)
0811 02:47:38.123531 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:38.123544 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:38.123651 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:38.123885 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:38.123887 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:38.124070 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:38.124172 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:38.124177 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:38.124903 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:38.124905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:38.125466 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:38.125473 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:38.125964 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:38.125966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:38.126027 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:38.126546 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:38.126557 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:38.128444 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":100,"threads_started":1}
W20260811 02:47:38.128750 5816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:38.123228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:52151 (local address 127.21.236.190:40517)
0811 02:47:38.123409 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:38.123415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:38.123657 (+ 242us) server_negotiation.cc:407] Connection header received
0811 02:47:38.123726 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:38.123728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:38.123776 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:38.123847 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:38.124276 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:38.124800 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:38.125597 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:38.125843 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:38.125959 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:38.128617 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:52151: BlockingRecv error: recv got EOF from 127.21.236.130:52151 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 02:47:38 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:38 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:38.343793 5820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:38.338266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:40533)
0811 02:47:38.338570 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:38.338585 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:38.338682 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:38.338933 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:38.338935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:38.339216 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:38.339321 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:38.339326 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:38.340080 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:38.340083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:38.340663 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:38.340672 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:38.341105 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:38.341108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:38.341179 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:38.341809 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:38.341822 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:38.343678 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":133,"threads_started":1}
W20260811 02:47:38.343932 5816 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:38.338364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:40533 (local address 127.21.236.190:40517)
0811 02:47:38.338505 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:38.338509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:38.338653 (+ 144us) server_negotiation.cc:407] Connection header received
0811 02:47:38.338761 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:38.338763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:38.338810 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:38.338880 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:38.339426 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:38.339960 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:38.340770 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:38.340983 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:38.341063 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:38.343810 (+ 2747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:40533: BlockingRecv error: recv got EOF from 127.21.236.131:40533 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 02:47:38 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:38 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:38.953986 5824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:38.947919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:47175)
0811 02:47:38.948310 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:38.948323 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:38.948426 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:38.948669 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:38.948671 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:38.948861 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:38.948958 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:38.948964 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:38.949771 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:38.949774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:38.950355 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:38.950363 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:38.950879 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:38.950882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:38.950948 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:38.951583 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:38.951595 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:38.953838 (+ 2243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":208,"threads_started":1}
W20260811 02:47:38.954259 5825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:38.948031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:47175 (local address 127.21.236.190:40517)
0811 02:47:38.948282 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:38.948286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:38.948429 (+ 143us) server_negotiation.cc:407] Connection header received
0811 02:47:38.948493 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:38.948495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:38.948543 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:38.948624 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:38.949111 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:38.949667 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:38.950535 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:38.950769 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:38.950858 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:38.954091 (+ 3233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:47175: BlockingRecv error: recv got EOF from 127.21.236.129:47175 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":90,"threads_started":1}
Aug 11 02:47:39 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:39 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:39.135460 5827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:39.129473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:57095)
0811 02:47:39.129825 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:39.129842 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:39.130000 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:39.130300 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:39.130303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:39.130532 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:39.130631 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:39.130637 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:39.131487 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:39.131492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:39.132165 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:39.132175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:39.132651 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:39.132654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:39.132739 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:39.133277 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:39.133289 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:39.135338 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":106,"threads_started":1}
W20260811 02:47:39.135716 5825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:39.129686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:57095 (local address 127.21.236.190:40517)
0811 02:47:39.129831 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:39.129838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:39.129931 (+ 93us) server_negotiation.cc:407] Connection header received
0811 02:47:39.130127 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:39.130130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:39.130181 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:39.130277 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:39.130757 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:39.131378 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:39.132292 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:39.132534 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:39.132644 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:39.135579 (+ 2935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:57095: BlockingRecv error: recv got EOF from 127.21.236.130:57095 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 02:47:39 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:39 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:39.350485 5828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:39.344728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:56423)
0811 02:47:39.345069 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:39.345085 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:39.345189 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:39.345455 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:39.345457 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:39.345664 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:39.345759 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:39.345764 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:39.346507 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:39.346510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:39.347211 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:39.347221 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:39.347783 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:39.347785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:39.347848 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:39.348484 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:39.348505 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:39.350369 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":92,"threads_started":1}
W20260811 02:47:39.350622 5825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:39.344824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:56423 (local address 127.21.236.190:40517)
0811 02:47:39.344988 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:39.344992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:39.345160 (+ 168us) server_negotiation.cc:407] Connection header received
0811 02:47:39.345283 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:39.345285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:39.345335 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:39.345412 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:39.345863 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:39.346382 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:39.347339 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:39.347650 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:39.347762 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:39.350501 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:56423: BlockingRecv error: recv got EOF from 127.21.236.131:56423 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 02:47:39 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:39 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:39.966653 5831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:39.954936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:42033)
0811 02:47:39.955257 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:39.955273 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:39.955380 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:39.955863 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:39.955865 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:39.956058 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:39.956157 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:39.956163 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:39.957143 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:39.957146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:39.957746 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:39.957754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:39.958240 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:39.958243 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:39.958309 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:39.964360 (+ 6051us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:39.964377 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:39.966516 (+ 2139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":155,"threads_started":1}
W20260811 02:47:39.966950 5832 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:39.955154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:42033 (local address 127.21.236.190:40517)
0811 02:47:39.955631 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:39.955634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:39.955649 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:47:39.955689 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:39.955691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:39.955734 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:39.955820 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:39.956317 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:39.957010 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:39.957858 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:39.958123 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:39.958210 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:39.966787 (+ 8577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:42033: BlockingRecv error: recv got EOF from 127.21.236.129:42033 (error 108)
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":140,"threads_started":1}
Aug 11 02:47:40 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:40 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:40.142216 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:40.136308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:53415)
0811 02:47:40.136673 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:40.136688 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:40.136795 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:40.137003 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:40.137005 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:40.137196 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:40.137295 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:40.137300 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:40.138049 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:40.138051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:40.138630 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:40.138639 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:40.139276 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:40.139280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:40.139372 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:40.140014 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:40.140029 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:40.142075 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":146,"threads_started":1}
W20260811 02:47:40.142351 5832 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:40.136370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:53415 (local address 127.21.236.190:40517)
0811 02:47:40.136529 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:40.136534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:40.136766 (+ 232us) server_negotiation.cc:407] Connection header received
0811 02:47:40.136863 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:40.136865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:40.136923 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:40.136993 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:40.137397 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:40.137949 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:40.138776 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:40.139130 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:40.139239 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:40.142224 (+ 2985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:53415: BlockingRecv error: recv got EOF from 127.21.236.130:53415 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 02:47:40 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:40 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.131@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:40.357658 5835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:40.351353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.131:34969)
0811 02:47:40.351746 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:40.351764 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:40.351920 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:40.352212 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:40.352215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:40.352431 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:40.352531 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:40.352536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:40.353313 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:40.353318 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:40.353935 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:40.353944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:40.354448 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:40.354471 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:40.354567 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:40.355270 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:40.355282 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:40.357454 (+ 2172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":99,"threads_started":1}
W20260811 02:47:40.357705 5832 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:40.351554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:34969 (local address 127.21.236.190:40517)
0811 02:47:40.351739 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:40.351744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:40.351854 (+ 110us) server_negotiation.cc:407] Connection header received
0811 02:47:40.352037 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:40.352041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:40.352096 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:40.352188 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:40.352658 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:40.353198 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:40.354069 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:40.354321 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:40.354426 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:40.357588 (+ 3162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.131:34969: BlockingRecv error: recv got EOF from 127.21.236.131:34969 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 11 02:47:40 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:40 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.129@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:40.973424 5836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:40.967631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.129:47239)
0811 02:47:40.968009 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:40.968024 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:40.968130 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:40.968401 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:40.968405 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:40.968608 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:40.968714 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:40.968720 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:40.969510 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:40.969513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:40.970136 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:40.970145 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:40.970599 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:40.970603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:40.970673 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:40.971301 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:40.971323 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:40.973309 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":117,"threads_started":1}
W20260811 02:47:40.973572 5837 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:40.967765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:47239 (local address 127.21.236.190:40517)
0811 02:47:40.968011 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:40.968014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:40.968086 (+ 72us) server_negotiation.cc:407] Connection header received
0811 02:47:40.968217 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:40.968219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:40.968265 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:40.968346 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:40.968819 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:40.969393 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:40.970247 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:40.970470 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:40.970553 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:40.973443 (+ 2890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.129:47239: BlockingRecv error: recv got EOF from 127.21.236.129:47239 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":65,"threads_started":1}
Aug 11 02:47:41 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 02:47:41 dist-test-slave-n326 krb5kdc[3917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.236.130@KRBTEST.COM for oryx/127.21.236.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 02:47:41.149093 5838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:41.143293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40517 (local address 127.21.236.130:32769)
0811 02:47:41.143618 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:41.143633 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:41.143732 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:41.144021 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:41.144024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:41.144228 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:41.144329 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:41.144335 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:41.145160 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:41.145163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:41.145748 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:41.145756 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:41.146220 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:41.146223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:41.146289 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:41.146882 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:41.146895 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:41.148968 (+ 2073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.236.190:40517: Server oryx/127.21.236.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":99,"threads_started":1}
W20260811 02:47:41.149276 5837 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:41.143471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:32769 (local address 127.21.236.190:40517)
0811 02:47:41.143650 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:41.143654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:41.143687 (+ 33us) server_negotiation.cc:407] Connection header received
0811 02:47:41.143822 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:41.143824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:41.143867 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:41.143946 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:41.144489 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:41.145043 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:41.145866 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:41.146105 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:41.146187 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:41.149136 (+ 2949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.236.130:32769: BlockingRecv error: recv got EOF from 127.21.236.130:32769 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 02:47:41.256631 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 4980
I20260811 02:47:41.273905 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 5257
I20260811 02:47:41.303264 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 5495
I20260811 02:47:41.314909 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 4859
2026-08-11T02:47:41Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18403 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T02:47:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:41Z Disabled control of system clock
I20260811 02:47:41.353794 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:39199
--webserver_interface=127.21.236.190
--webserver_port=0
--builtin_ntp_servers=127.21.236.148:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:39199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 02:47:41.465191 5847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:41.465442 5847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:41.465476 5847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:41.469441 5847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:41.469498 5847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:41.469514 5847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:41.469532 5847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:41.469547 5847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:47:41.474390 5847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:39199
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:39199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:41.475508 5847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:41.476845 5847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:47:41.485782 5847 server_base.cc:1034] running on GCE node
W20260811 02:47:41.485843 5861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:41.485836 5858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:41.485935 5864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:41.486618 5847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:41.487859 5847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:41.489037 5847 hybrid_clock.cc:648] HybridClock initialized: now 1786416461489000 us; error 39 us; skew 500 ppm
I20260811 02:47:41.491137 5847 webserver.cc:460] Webserver started at http://127.21.236.190:32927/ using document root <none> and password file <none>
I20260811 02:47:41.491768 5847 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:41.491838 5847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:41.492132 5847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:41.494356 5847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "98f3071c98284cf3a0a2e76290f14074"
format_stamp: "Formatted at 2026-08-11 02:47:41 on dist-test-slave-n326"
server_key: "34c5482740d7e29470be888e969ef8d2"
server_key_iv: "5b8fd64364e8b971e7974935da9edeca"
server_key_version: "encryptionkey@0"
I20260811 02:47:41.494966 5847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "98f3071c98284cf3a0a2e76290f14074"
format_stamp: "Formatted at 2026-08-11 02:47:41 on dist-test-slave-n326"
server_key: "34c5482740d7e29470be888e969ef8d2"
server_key_iv: "5b8fd64364e8b971e7974935da9edeca"
server_key_version: "encryptionkey@0"
I20260811 02:47:41.498782 5847 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:47:41.501368 5877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:41.502650 5847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:47:41.502787 5847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "98f3071c98284cf3a0a2e76290f14074"
format_stamp: "Formatted at 2026-08-11 02:47:41 on dist-test-slave-n326"
server_key: "34c5482740d7e29470be888e969ef8d2"
server_key_iv: "5b8fd64364e8b971e7974935da9edeca"
server_key_version: "encryptionkey@0"
I20260811 02:47:41.502905 5847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:41.545236 5847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:41.546043 5847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:41.546268 5847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:41.554277 5847 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:39199
I20260811 02:47:41.554288 5966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:39199 every 8 connection(s)
I20260811 02:47:41.555369 5847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:41.558717 5967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:41.560186 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 5847
I20260811 02:47:41.560338 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:47:41.560672 22450 external_mini_cluster.cc:1428] Setting key 1eef620d6afdc8be5a94a2a4bcb4d2f8
I20260811 02:47:41.565287 5967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074: Bootstrap starting.
I20260811 02:47:41.567562 5967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:41.568527 5967 log.cc:824] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:41.570741 5973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:41.562218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38618 (local address 127.21.236.190:39199)
0811 02:47:41.562943 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:41.562960 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:41.562999 (+ 39us) server_negotiation.cc:407] Connection header received
0811 02:47:41.563770 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:41.563788 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:41.564195 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:41.564558 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:41.565016 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:41.566123 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:41.567141 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:41.567646 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:41.567834 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:41.568748 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:41.568754 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:41.568770 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:41.569214 (+ 444us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:41.569269 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:41.569320 (+ 51us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:41.569424 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:41.569569 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 02:47:41.569826 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":142,"threads_started":1}
I20260811 02:47:41.570796 5967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074: No bootstrap required, opened a new log
I20260811 02:47:41.574693 5967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f3071c98284cf3a0a2e76290f14074" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39199 } }
I20260811 02:47:41.574985 5967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:41.575040 5967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98f3071c98284cf3a0a2e76290f14074, State: Initialized, Role: FOLLOWER
I20260811 02:47:41.575488 5967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f3071c98284cf3a0a2e76290f14074" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39199 } }
I20260811 02:47:41.575628 5967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:41.575671 5967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:41.575742 5967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:41.576566 5967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f3071c98284cf3a0a2e76290f14074" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39199 } }
I20260811 02:47:41.576843 5967 leader_election.cc:302] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98f3071c98284cf3a0a2e76290f14074; no voters:
I20260811 02:47:41.577092 5967 leader_election.cc:288] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:47:41.577301 5975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:41.577610 5975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [term 1 LEADER]: Becoming Leader. State: Replica: 98f3071c98284cf3a0a2e76290f14074, State: Running, Role: LEADER
I20260811 02:47:41.577971 5975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f3071c98284cf3a0a2e76290f14074" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39199 } }
I20260811 02:47:41.578325 5967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:41.579681 5976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98f3071c98284cf3a0a2e76290f14074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f3071c98284cf3a0a2e76290f14074" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39199 } } }
I20260811 02:47:41.579813 5976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:41.580324 5977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98f3071c98284cf3a0a2e76290f14074. Latest consensus state: current_term: 1 leader_uuid: "98f3071c98284cf3a0a2e76290f14074" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f3071c98284cf3a0a2e76290f14074" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39199 } } }
I20260811 02:47:41.580497 5977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:41.580797 5987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:41.583873 5987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:41.589883 5987 catalog_manager.cc:1347] Generated new cluster ID: eccfbd69ddc84c27af1e195989ee793a
I20260811 02:47:41.589962 5987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:41.600713 5987 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:47:41.602062 5987 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:41.615602 5987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98f3071c98284cf3a0a2e76290f14074: Generated new TSK 0
I20260811 02:47:41.616493 5987 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:47:41.624619 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39199
--builtin_ntp_servers=127.21.236.148:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 02:47:41.740856 5999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:41.741140 5999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:41.741178 5999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:41.745545 5999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:41.745610 5999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:41.745666 5999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:47:41.750514 5999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:41.751741 5999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:41.753165 5999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:41.762991 6008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:41.763008 6009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:41.763116 5999 server_base.cc:1034] running on GCE node
W20260811 02:47:41.763008 6012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:41.763789 5999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:41.764477 5999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:41.765671 5999 hybrid_clock.cc:648] HybridClock initialized: now 1786416461765658 us; error 45 us; skew 500 ppm
I20260811 02:47:41.767655 5999 webserver.cc:460] Webserver started at http://127.21.236.129:41395/ using document root <none> and password file <none>
I20260811 02:47:41.768272 5999 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:41.768375 5999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:41.768630 5999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:41.770768 5999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "422cbfb9900845ae9461e336f9b03aab"
format_stamp: "Formatted at 2026-08-11 02:47:41 on dist-test-slave-n326"
server_key: "8c06c34d1bbd93e37041c37ffd1ee5df"
server_key_iv: "7225d74e7018cb71dd8c1b3b39f4a615"
server_key_version: "encryptionkey@0"
I20260811 02:47:41.771385 5999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "422cbfb9900845ae9461e336f9b03aab"
format_stamp: "Formatted at 2026-08-11 02:47:41 on dist-test-slave-n326"
server_key: "8c06c34d1bbd93e37041c37ffd1ee5df"
server_key_iv: "7225d74e7018cb71dd8c1b3b39f4a615"
server_key_version: "encryptionkey@0"
I20260811 02:47:41.775368 5999 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:47:41.778187 6030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:41.779752 5999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:47:41.779899 5999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "422cbfb9900845ae9461e336f9b03aab"
format_stamp: "Formatted at 2026-08-11 02:47:41 on dist-test-slave-n326"
server_key: "8c06c34d1bbd93e37041c37ffd1ee5df"
server_key_iv: "7225d74e7018cb71dd8c1b3b39f4a615"
server_key_version: "encryptionkey@0"
I20260811 02:47:41.780046 5999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:41.792922 5999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:41.793644 5999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:41.793849 5999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:41.794445 5999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:41.795459 5999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:41.795540 5999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:41.795634 5999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:41.795684 5999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:41.804626 5999 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:39129
I20260811 02:47:41.804649 6231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:39129 every 8 connection(s)
I20260811 02:47:41.805531 5999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:47:41.812955 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 5999
I20260811 02:47:41.813057 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:47:41.813385 22450 external_mini_cluster.cc:1428] Setting key a62ce9673197b9c95a6be955d734cff5
I20260811 02:47:41.814865 5973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:41.807491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:40005 (local address 127.21.236.190:39199)
0811 02:47:41.807629 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:41.807633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:41.808722 (+ 1089us) server_negotiation.cc:407] Connection header received
0811 02:47:41.810052 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:41.810055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:41.810101 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:41.810176 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:41.811043 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:41.811657 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:41.812822 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:41.813087 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:41.813200 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:41.814377 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:41.814381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:41.814383 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:41.814395 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:41.814413 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:41.814422 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:41.814502 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:41.814665 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 02:47:41.814729 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 02:47:41.815478 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39199
--builtin_ntp_servers=127.21.236.148:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 02:47:41.815824 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:41.807911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:39199 (local address 127.21.236.129:40005)
0811 02:47:41.808567 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:41.808616 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:41.809870 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:41.810343 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:41.810349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:41.810428 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:41.810904 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:41.810916 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:41.811776 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:41.811779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:41.812687 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:41.812700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:41.813259 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:41.813264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:41.813433 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:41.814106 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:47:41.814108 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:41.814148 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:47:41.814213 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:41.814504 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:41.814517 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:41.814523 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:47:41.814665 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 02:47:41.814955 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":129,"threads_started":1}
I20260811 02:47:41.817203 6232 heartbeater.cc:347] Connected to a master server at 127.21.236.190:39199
I20260811 02:47:41.817458 6232 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:41.817978 6232 heartbeater.cc:511] Master 127.21.236.190:39199 requested a full tablet report, sending...
I20260811 02:47:41.819662 5899 ts_manager.cc:194] Registered new tserver with Master: 422cbfb9900845ae9461e336f9b03aab (127.21.236.129:39129)
I20260811 02:47:41.821389 5899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.236.129:40005
W20260811 02:47:41.930397 6238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:41.930660 6238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:41.930696 6238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:41.934532 6238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:41.934587 6238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:41.934638 6238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:47:41.939164 6238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:41.940155 6238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:41.941501 6238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:41.951153 6247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:41.951191 6250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:41.951153 6246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:41.951371 6238 server_base.cc:1034] running on GCE node
I20260811 02:47:41.951864 6238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:41.952529 6238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:41.953720 6238 hybrid_clock.cc:648] HybridClock initialized: now 1786416461953703 us; error 28 us; skew 500 ppm
I20260811 02:47:41.955596 6238 webserver.cc:460] Webserver started at http://127.21.236.130:41343/ using document root <none> and password file <none>
I20260811 02:47:41.956182 6238 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:41.956277 6238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:41.956512 6238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:41.958590 6238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "03cfe18abf85422988c5fb890a8076d8"
format_stamp: "Formatted at 2026-08-11 02:47:41 on dist-test-slave-n326"
server_key: "aae6a9ec33b07182dc8b872aaf543e3e"
server_key_iv: "3491eb004e15dbd02a09d014f8fd8aa9"
server_key_version: "encryptionkey@0"
I20260811 02:47:41.959241 6238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "03cfe18abf85422988c5fb890a8076d8"
format_stamp: "Formatted at 2026-08-11 02:47:41 on dist-test-slave-n326"
server_key: "aae6a9ec33b07182dc8b872aaf543e3e"
server_key_iv: "3491eb004e15dbd02a09d014f8fd8aa9"
server_key_version: "encryptionkey@0"
I20260811 02:47:41.963153 6238 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:47:41.965781 6258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:41.967207 6238 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 02:47:41.967351 6238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "03cfe18abf85422988c5fb890a8076d8"
format_stamp: "Formatted at 2026-08-11 02:47:41 on dist-test-slave-n326"
server_key: "aae6a9ec33b07182dc8b872aaf543e3e"
server_key_iv: "3491eb004e15dbd02a09d014f8fd8aa9"
server_key_version: "encryptionkey@0"
I20260811 02:47:41.967473 6238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:41.999787 6238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:42.000552 6238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:42.000767 6238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:42.001466 6238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:42.002403 6238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:42.002451 6238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:42.002490 6238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:42.002522 6238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:42.012205 6238 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:44123
I20260811 02:47:42.012236 6451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:44123 every 8 connection(s)
I20260811 02:47:42.013149 6238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:47:42.013899 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 6238
I20260811 02:47:42.014010 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:47:42.014374 22450 external_mini_cluster.cc:1428] Setting key 80cc83c6199a5ba8f6a1ad00857e1414
I20260811 02:47:42.016430 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39199
--builtin_ntp_servers=127.21.236.148:35783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 02:47:42.023885 5973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:42.015216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:58941 (local address 127.21.236.190:39199)
0811 02:47:42.015351 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:42.015355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:42.016666 (+ 1311us) server_negotiation.cc:407] Connection header received
0811 02:47:42.018182 (+ 1516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:42.018185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:42.018229 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:42.018299 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:42.019543 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:42.020384 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:42.021545 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:42.021974 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:42.022105 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:42.023264 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:42.023271 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:42.023277 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:42.023295 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:42.023321 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:42.023337 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:42.023427 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:42.023591 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 02:47:42.023669 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:47:42.024681 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:42.015720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:39199 (local address 127.21.236.130:58941)
0811 02:47:42.016518 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:42.016572 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:42.017925 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:42.018497 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:42.018509 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:42.018621 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:42.019361 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:42.019383 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:42.020526 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:42.020529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:42.021425 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:42.021435 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:42.022125 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:42.022128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:42.022270 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:42.022928 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:47:42.022929 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:42.022970 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:47:42.023040 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:42.023432 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:42.023439 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:42.023443 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 02:47:42.023534 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 02:47:42.023767 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":170,"threads_started":1}
I20260811 02:47:42.025877 6452 heartbeater.cc:347] Connected to a master server at 127.21.236.190:39199
I20260811 02:47:42.026136 6452 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:42.026676 6452 heartbeater.cc:511] Master 127.21.236.190:39199 requested a full tablet report, sending...
I20260811 02:47:42.027808 5899 ts_manager.cc:194] Registered new tserver with Master: 03cfe18abf85422988c5fb890a8076d8 (127.21.236.130:44123)
I20260811 02:47:42.028699 5899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.236.130:58941
W20260811 02:47:42.131716 6456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:42.131974 6456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:42.132011 6456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:42.136042 6456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:42.136103 6456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:42.136157 6456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:47:42.140738 6456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:35783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:42.141782 6456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:42.143172 6456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:42.152877 6467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:42.152976 6456 server_base.cc:1034] running on GCE node
W20260811 02:47:42.152926 6471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:42.152901 6475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:42.153439 6456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:42.154063 6456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:42.155222 6456 hybrid_clock.cc:648] HybridClock initialized: now 1786416462155184 us; error 52 us; skew 500 ppm
I20260811 02:47:42.156996 6456 webserver.cc:460] Webserver started at http://127.21.236.131:46183/ using document root <none> and password file <none>
I20260811 02:47:42.157593 6456 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:42.157660 6456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:42.157877 6456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:42.160009 6456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "879362af0ec74839b55a4422d675a810"
format_stamp: "Formatted at 2026-08-11 02:47:42 on dist-test-slave-n326"
server_key: "00be01b846091c6017bbe94eb85259fc"
server_key_iv: "1060a47a5bf09c94fa8e33400b43d830"
server_key_version: "encryptionkey@0"
I20260811 02:47:42.160588 6456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "879362af0ec74839b55a4422d675a810"
format_stamp: "Formatted at 2026-08-11 02:47:42 on dist-test-slave-n326"
server_key: "00be01b846091c6017bbe94eb85259fc"
server_key_iv: "1060a47a5bf09c94fa8e33400b43d830"
server_key_version: "encryptionkey@0"
I20260811 02:47:42.164312 6456 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:47:42.166908 6485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:42.168467 6456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:47:42.168612 6456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "879362af0ec74839b55a4422d675a810"
format_stamp: "Formatted at 2026-08-11 02:47:42 on dist-test-slave-n326"
server_key: "00be01b846091c6017bbe94eb85259fc"
server_key_iv: "1060a47a5bf09c94fa8e33400b43d830"
server_key_version: "encryptionkey@0"
I20260811 02:47:42.168733 6456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:42.182154 6456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:42.182876 6456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:42.183147 6456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:42.183823 6456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:42.184846 6456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:42.184928 6456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:42.184996 6456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:42.185048 6456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:42.193857 6456 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:35803
I20260811 02:47:42.193888 6690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:35803 every 8 connection(s)
I20260811 02:47:42.194796 6456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:47:42.204667 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 6456
I20260811 02:47:42.204787 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:47:42.205145 22450 external_mini_cluster.cc:1428] Setting key 2a942b926c23364a3d91c364927873d6
I20260811 02:47:42.205488 5973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:42.196744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:58373 (local address 127.21.236.190:39199)
0811 02:47:42.196919 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:42.196924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:42.197994 (+ 1070us) server_negotiation.cc:407] Connection header received
0811 02:47:42.199890 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:42.199894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:42.199979 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:42.200087 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:42.201075 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:42.201678 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:42.202966 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:42.203291 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:42.203419 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:42.204809 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:42.204829 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:42.204833 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:42.204850 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:42.204872 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:42.204882 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:42.204976 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:42.205225 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 02:47:42.205297 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 02:47:42.207002 6697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:42.197146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:39199 (local address 127.21.236.131:58373)
0811 02:47:42.197833 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:42.197877 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:42.199575 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:42.200291 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:42.200299 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:42.200388 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:42.200911 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:42.200924 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:42.201821 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:42.201825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:42.202812 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:42.202821 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:42.203471 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:42.203476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:42.203658 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:42.204466 (+ 808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:47:42.204470 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:42.204527 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:47:42.204624 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:42.205037 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:42.205049 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:42.205056 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 02:47:42.205664 (+ 608us) client_negotiation.cc:240] Negotiation successful
0811 02:47:42.205969 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":101,"threads_started":1}
I20260811 02:47:42.209081 6691 heartbeater.cc:347] Connected to a master server at 127.21.236.190:39199
I20260811 02:47:42.209329 6691 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:42.210016 6691 heartbeater.cc:511] Master 127.21.236.190:39199 requested a full tablet report, sending...
I20260811 02:47:42.211184 5899 ts_manager.cc:194] Registered new tserver with Master: 879362af0ec74839b55a4422d675a810 (127.21.236.131:35803)
I20260811 02:47:42.211854 5899 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.236.131:58373
I20260811 02:47:42.220563 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 02:47:42.223698 5973 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:42.222535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38634 (local address 127.21.236.190:39199)
0811 02:47:42.222717 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:42.222722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:42.222862 (+ 140us) server_negotiation.cc:407] Connection header received
0811 02:47:42.223038 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:42.223041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:42.223111 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:42.223200 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:42.223552 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38634: BlockingRecv error: recv got EOF from 127.0.0.1:38634 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 02:47:42.223912 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 5999
I20260811 02:47:42.240772 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 6238
I20260811 02:47:42.267495 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 6456
I20260811 02:47:42.280457 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 5847
2026-08-11T02:47:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (966 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T02:47:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:42Z Disabled control of system clock
I20260811 02:47:42.321023 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:33261
--webserver_interface=127.21.236.190
--webserver_port=0
--builtin_ntp_servers=127.21.236.148:33781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:33261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 02:47:42.434015 6711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:42.434351 6711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:42.434430 6711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:42.438453 6711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:42.438555 6711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:42.438585 6711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:42.438623 6711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:42.438679 6711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:47:42.443485 6711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:33261
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:33261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:42.444629 6711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:42.446038 6711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:42.454312 6719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:42.454349 6725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:42.454342 6721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:42.454589 6711 server_base.cc:1034] running on GCE node
I20260811 02:47:42.455281 6711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:42.456413 6711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:42.457609 6711 hybrid_clock.cc:648] HybridClock initialized: now 1786416462457571 us; error 48 us; skew 500 ppm
I20260811 02:47:42.459625 6711 webserver.cc:460] Webserver started at http://127.21.236.190:37787/ using document root <none> and password file <none>
I20260811 02:47:42.460244 6711 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:42.460346 6711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:42.460599 6711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:42.462728 6711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "4e4cdbce93c74789a06583e499f35eee"
format_stamp: "Formatted at 2026-08-11 02:47:42 on dist-test-slave-n326"
server_key: "8720c6d865f06c64b52004b61720ab83"
server_key_iv: "a1fd4c6827591fc5ff988c90dddec61c"
server_key_version: "encryptionkey@0"
I20260811 02:47:42.463335 6711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "4e4cdbce93c74789a06583e499f35eee"
format_stamp: "Formatted at 2026-08-11 02:47:42 on dist-test-slave-n326"
server_key: "8720c6d865f06c64b52004b61720ab83"
server_key_iv: "a1fd4c6827591fc5ff988c90dddec61c"
server_key_version: "encryptionkey@0"
I20260811 02:47:42.467226 6711 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:47:42.469725 6734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:42.470908 6711 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:47:42.471122 6711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "4e4cdbce93c74789a06583e499f35eee"
format_stamp: "Formatted at 2026-08-11 02:47:42 on dist-test-slave-n326"
server_key: "8720c6d865f06c64b52004b61720ab83"
server_key_iv: "a1fd4c6827591fc5ff988c90dddec61c"
server_key_version: "encryptionkey@0"
I20260811 02:47:42.471256 6711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:42.475893 6711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:42.476577 6711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:42.476760 6711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:42.483745 6711 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:33261
I20260811 02:47:42.483791 6828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:33261 every 8 connection(s)
I20260811 02:47:42.484714 6711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:42.487401 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 6711
I20260811 02:47:42.487569 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:47:42.487884 22450 external_mini_cluster.cc:1428] Setting key ad0aecf24fda464e9f0a2e9c3d0a81a9
I20260811 02:47:42.488044 6829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:42.494045 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:42.489394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55244 (local address 127.21.236.190:33261)
0811 02:47:42.490113 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:42.490132 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:42.490177 (+ 45us) server_negotiation.cc:407] Connection header received
0811 02:47:42.490998 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:42.491023 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:42.491508 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:42.491875 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:42.492605 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:42.492611 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:42.492624 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:42.492635 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:42.492700 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:42.492722 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:42.492857 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:42.492928 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 02:47:42.493182 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":132,"threads_started":1}
I20260811 02:47:42.495647 6829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee: Bootstrap starting.
I20260811 02:47:42.497816 6829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:42.498546 6829 log.cc:824] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:42.500563 6829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee: No bootstrap required, opened a new log
I20260811 02:47:42.504144 6829 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4cdbce93c74789a06583e499f35eee" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33261 } }
I20260811 02:47:42.504381 6829 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:42.504464 6829 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e4cdbce93c74789a06583e499f35eee, State: Initialized, Role: FOLLOWER
I20260811 02:47:42.504886 6829 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4cdbce93c74789a06583e499f35eee" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33261 } }
I20260811 02:47:42.505035 6829 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:42.505124 6829 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:42.505224 6829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:42.506101 6829 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4cdbce93c74789a06583e499f35eee" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33261 } }
I20260811 02:47:42.506439 6829 leader_election.cc:302] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e4cdbce93c74789a06583e499f35eee; no voters:
I20260811 02:47:42.506749 6829 leader_election.cc:288] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:47:42.506896 6836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:42.507155 6836 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [term 1 LEADER]: Becoming Leader. State: Replica: 4e4cdbce93c74789a06583e499f35eee, State: Running, Role: LEADER
I20260811 02:47:42.507457 6836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4cdbce93c74789a06583e499f35eee" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33261 } }
I20260811 02:47:42.508018 6829 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:42.509114 6839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e4cdbce93c74789a06583e499f35eee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4cdbce93c74789a06583e499f35eee" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33261 } } }
I20260811 02:47:42.509263 6839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:42.509609 6840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e4cdbce93c74789a06583e499f35eee. Latest consensus state: current_term: 1 leader_uuid: "4e4cdbce93c74789a06583e499f35eee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4cdbce93c74789a06583e499f35eee" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33261 } } }
I20260811 02:47:42.509734 6840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:42.510039 6851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:42.512759 6851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:42.518400 6851 catalog_manager.cc:1347] Generated new cluster ID: 00fd56d72b9d40f797b7c24677f5a52c
I20260811 02:47:42.518477 6851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:42.546609 6851 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:47:42.546967 6851 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:42.558915 6851 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e4cdbce93c74789a06583e499f35eee: Generated new TSK 0
I20260811 02:47:42.559733 6851 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:47:42.575266 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:33261
--builtin_ntp_servers=127.21.236.148:33781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 02:47:42.696179 6865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:42.696462 6865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:42.696504 6865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:42.700523 6865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:42.700582 6865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:42.700637 6865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:47:42.705314 6865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:33261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:42.706362 6865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:42.707756 6865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:42.718003 6872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:42.718006 6873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:42.718003 6876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:42.718286 6865 server_base.cc:1034] running on GCE node
I20260811 02:47:42.718735 6865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:42.719512 6865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:42.720749 6865 hybrid_clock.cc:648] HybridClock initialized: now 1786416462720725 us; error 32 us; skew 500 ppm
I20260811 02:47:42.722856 6865 webserver.cc:460] Webserver started at http://127.21.236.129:42341/ using document root <none> and password file <none>
I20260811 02:47:42.723577 6865 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:42.723646 6865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:42.723824 6865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:42.725894 6865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "3ab97e4135c541b9803fa85017db1070"
format_stamp: "Formatted at 2026-08-11 02:47:42 on dist-test-slave-n326"
server_key: "252b0ba4e9942f40da023d1f3f43dfc4"
server_key_iv: "84edeac969c871ae3781e4130e12cc87"
server_key_version: "encryptionkey@0"
I20260811 02:47:42.726420 6865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "3ab97e4135c541b9803fa85017db1070"
format_stamp: "Formatted at 2026-08-11 02:47:42 on dist-test-slave-n326"
server_key: "252b0ba4e9942f40da023d1f3f43dfc4"
server_key_iv: "84edeac969c871ae3781e4130e12cc87"
server_key_version: "encryptionkey@0"
I20260811 02:47:42.730158 6865 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:47:42.732836 6892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:42.734130 6865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:47:42.734230 6865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "3ab97e4135c541b9803fa85017db1070"
format_stamp: "Formatted at 2026-08-11 02:47:42 on dist-test-slave-n326"
server_key: "252b0ba4e9942f40da023d1f3f43dfc4"
server_key_iv: "84edeac969c871ae3781e4130e12cc87"
server_key_version: "encryptionkey@0"
I20260811 02:47:42.734323 6865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:42.738957 6865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:42.739609 6865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:42.739758 6865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:42.740259 6865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:42.741175 6865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:42.741225 6865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:42.741264 6865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:42.741281 6865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:42.750757 6865 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:44023
I20260811 02:47:42.750777 7113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:44023 every 8 connection(s)
I20260811 02:47:42.751837 6865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:47:42.753736 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 6865
I20260811 02:47:42.753880 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:47:42.754211 22450 external_mini_cluster.cc:1428] Setting key 0f01218ec3be056af02817351569f5ee
I20260811 02:47:42.756225 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:33261
--builtin_ntp_servers=127.21.236.148:33781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 02:47:42.758342 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:42.753728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:53195 (local address 127.21.236.190:33261)
0811 02:47:42.753911 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:42.753916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:42.755046 (+ 1130us) server_negotiation.cc:407] Connection header received
0811 02:47:42.756450 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:42.756453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:42.756499 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:42.756544 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:42.757826 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:42.757830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:42.757832 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:42.757844 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:42.757865 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:42.757881 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:42.757990 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:42.758160 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 02:47:42.758206 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 02:47:42.759320 7118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:42.754129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:33261 (local address 127.21.236.129:53195)
0811 02:47:42.754883 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:42.754936 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:42.756186 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:42.756846 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:42.756853 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:42.756937 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:42.757569 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:47:42.757572 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:42.757614 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:47:42.757682 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:42.757982 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:42.757991 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:42.757999 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 02:47:42.758195 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 02:47:42.758443 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":179,"threads_started":1}
I20260811 02:47:42.760654 7114 heartbeater.cc:347] Connected to a master server at 127.21.236.190:33261
I20260811 02:47:42.760936 7114 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:42.761494 7114 heartbeater.cc:511] Master 127.21.236.190:33261 requested a full tablet report, sending...
I20260811 02:47:42.763260 6763 ts_manager.cc:194] Registered new tserver with Master: 3ab97e4135c541b9803fa85017db1070 (127.21.236.129:44023)
W20260811 02:47:42.872959 7120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:42.873239 7120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:42.873281 7120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:42.877725 7120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:42.877791 7120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:42.877851 7120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:47:42.882860 7120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:33261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:42.884030 7120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:42.885453 7120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:42.895164 7135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:42.895172 7136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:42.895283 7120 server_base.cc:1034] running on GCE node
W20260811 02:47:42.895174 7141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:42.895799 7120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:42.896415 7120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:42.897625 7120 hybrid_clock.cc:648] HybridClock initialized: now 1786416462897613 us; error 30 us; skew 500 ppm
I20260811 02:47:42.899585 7120 webserver.cc:460] Webserver started at http://127.21.236.130:44663/ using document root <none> and password file <none>
I20260811 02:47:42.900388 7120 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:42.900486 7120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:42.900733 7120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:42.903684 7120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "24c18b8ffb524115a1aaf0fa778d0408"
format_stamp: "Formatted at 2026-08-11 02:47:42 on dist-test-slave-n326"
server_key: "b8b8da61df38bac9b58965b4bf204825"
server_key_iv: "1cb26444a2d58a9b3a79618fcefad4f3"
server_key_version: "encryptionkey@0"
I20260811 02:47:42.904250 7120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "24c18b8ffb524115a1aaf0fa778d0408"
format_stamp: "Formatted at 2026-08-11 02:47:42 on dist-test-slave-n326"
server_key: "b8b8da61df38bac9b58965b4bf204825"
server_key_iv: "1cb26444a2d58a9b3a79618fcefad4f3"
server_key_version: "encryptionkey@0"
I20260811 02:47:42.908211 7120 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:47:42.911140 7154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:42.912474 7120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:47:42.912614 7120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "24c18b8ffb524115a1aaf0fa778d0408"
format_stamp: "Formatted at 2026-08-11 02:47:42 on dist-test-slave-n326"
server_key: "b8b8da61df38bac9b58965b4bf204825"
server_key_iv: "1cb26444a2d58a9b3a79618fcefad4f3"
server_key_version: "encryptionkey@0"
I20260811 02:47:42.912740 7120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:42.917459 7120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:42.918077 7120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:42.918260 7120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:42.918807 7120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:42.919777 7120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:42.919826 7120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:42.919890 7120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:42.919921 7120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:42.928835 7120 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:35537
I20260811 02:47:42.928877 7359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:35537 every 8 connection(s)
I20260811 02:47:42.929716 7120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:47:42.934239 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 7120
I20260811 02:47:42.934389 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:47:42.934716 22450 external_mini_cluster.cc:1428] Setting key 9292f04bf51290e39fa34f9e950a620f
I20260811 02:47:42.935966 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:42.931596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:51811 (local address 127.21.236.190:33261)
0811 02:47:42.931748 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:42.931751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:42.932792 (+ 1041us) server_negotiation.cc:407] Connection header received
0811 02:47:42.934099 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:42.934103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:42.934149 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:42.934239 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:42.935476 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:42.935479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:42.935482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:42.935495 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:42.935516 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:42.935533 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:42.935642 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:42.935791 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 02:47:42.935836 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:47:42.936997 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:42.931990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:33261 (local address 127.21.236.130:51811)
0811 02:47:42.932646 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:42.932686 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:42.933879 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:42.934447 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:42.934453 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:42.934575 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:42.935217 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:47:42.935219 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:42.935259 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:47:42.935324 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:42.935627 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:42.935637 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:42.935642 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 02:47:42.935822 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 02:47:42.936060 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":119,"threads_started":1}
I20260811 02:47:42.936983 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:33261
--builtin_ntp_servers=127.21.236.148:33781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 02:47:42.938192 7361 heartbeater.cc:347] Connected to a master server at 127.21.236.190:33261
I20260811 02:47:42.938462 7361 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:42.939008 7361 heartbeater.cc:511] Master 127.21.236.190:33261 requested a full tablet report, sending...
I20260811 02:47:42.940228 6763 ts_manager.cc:194] Registered new tserver with Master: 24c18b8ffb524115a1aaf0fa778d0408 (127.21.236.130:35537)
W20260811 02:47:43.057817 7367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:43.058082 7367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:43.058123 7367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:43.062402 7367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:43.062474 7367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:43.062531 7367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:47:43.067570 7367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:33261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:43.068653 7367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:43.070014 7367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:43.079990 7373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:43.080021 7375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:43.079950 7372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:43.080298 7367 server_base.cc:1034] running on GCE node
I20260811 02:47:43.080739 7367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:43.081416 7367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:43.082645 7367 hybrid_clock.cc:648] HybridClock initialized: now 1786416463082626 us; error 36 us; skew 500 ppm
I20260811 02:47:43.084822 7367 webserver.cc:460] Webserver started at http://127.21.236.131:34929/ using document root <none> and password file <none>
I20260811 02:47:43.085505 7367 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:43.085620 7367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:43.085887 7367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:43.088238 7367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "7ae46200877949e4ab794c6bd9bb1586"
format_stamp: "Formatted at 2026-08-11 02:47:43 on dist-test-slave-n326"
server_key: "a3b0e885a3f201a718ab36726820428c"
server_key_iv: "a4e0e32b17ddc53e7c7b075a9a59898e"
server_key_version: "encryptionkey@0"
I20260811 02:47:43.088853 7367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "7ae46200877949e4ab794c6bd9bb1586"
format_stamp: "Formatted at 2026-08-11 02:47:43 on dist-test-slave-n326"
server_key: "a3b0e885a3f201a718ab36726820428c"
server_key_iv: "a4e0e32b17ddc53e7c7b075a9a59898e"
server_key_version: "encryptionkey@0"
I20260811 02:47:43.092962 7367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:47:43.095770 7387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:43.097038 7367 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:47:43.097208 7367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "7ae46200877949e4ab794c6bd9bb1586"
format_stamp: "Formatted at 2026-08-11 02:47:43 on dist-test-slave-n326"
server_key: "a3b0e885a3f201a718ab36726820428c"
server_key_iv: "a4e0e32b17ddc53e7c7b075a9a59898e"
server_key_version: "encryptionkey@0"
I20260811 02:47:43.097339 7367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:43.102324 7367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:43.102969 7367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:43.103230 7367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:43.103820 7367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:43.104786 7367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:43.104866 7367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:43.104941 7367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:43.104998 7367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:43.115020 7367 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:42289
I20260811 02:47:43.115041 7585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:42289 every 8 connection(s)
I20260811 02:47:43.116083 7367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:47:43.122339 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:43.118100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:54917 (local address 127.21.236.190:33261)
0811 02:47:43.118243 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:43.118247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:43.119325 (+ 1078us) server_negotiation.cc:407] Connection header received
0811 02:47:43.120636 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:43.120639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:43.120679 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:43.120768 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:43.121863 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:43.121867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:43.121869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:47:43.121881 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:43.121899 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:47:43.121907 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:43.122001 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:43.122189 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 02:47:43.122227 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:47:43.123394 7588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:43.118502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:33261 (local address 127.21.236.131:54917)
0811 02:47:43.119180 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:43.119219 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:43.120385 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:43.120924 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:43.120932 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:43.121013 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:43.121610 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:47:43.121612 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:43.121652 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:47:43.121720 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:43.122032 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:43.122042 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:43.122048 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:47:43.122231 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 02:47:43.122533 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":116,"threads_started":1}
I20260811 02:47:43.124619 7586 heartbeater.cc:347] Connected to a master server at 127.21.236.190:33261
I20260811 02:47:43.124884 7586 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:43.125332 7586 heartbeater.cc:511] Master 127.21.236.190:33261 requested a full tablet report, sending...
I20260811 02:47:43.125913 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 7367
I20260811 02:47:43.126051 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:47:43.126425 22450 external_mini_cluster.cc:1428] Setting key 899ac2af89d82b8d32811c58420a68a6
I20260811 02:47:43.126652 6763 ts_manager.cc:194] Registered new tserver with Master: 7ae46200877949e4ab794c6bd9bb1586 (127.21.236.131:42289)
I20260811 02:47:43.128729 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 02:47:43.131728 6834 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:47:43.130771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55258 (local address 127.21.236.190:33261)
0811 02:47:43.130905 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:43.130909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:43.131114 (+ 205us) server_negotiation.cc:407] Connection header received
0811 02:47:43.131243 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:43.131245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:43.131276 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:43.131345 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:43.131598 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55258: BlockingRecv error: recv got EOF from 127.0.0.1:55258 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:47:43.132025 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 6865
I20260811 02:47:43.153100 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 7120
I20260811 02:47:43.168672 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 7367
I20260811 02:47:43.181674 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 6711
2026-08-11T02:47:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (897 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:47:43 dist-test-slave-n326 krb5kdc[7602](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:47:43 dist-test-slave-n326 krb5kdc[7602](info): set up 2 sockets
Aug 11 02:47:43 dist-test-slave-n326 krb5kdc[7602](info): commencing operation
krb5kdc: starting...
W20260811 02:47:44.506510 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:47:44 dist-test-slave-n326 krb5kdc[7602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:47:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:47:44.650914 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:41559
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:41559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:44.766702 7623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:44.767020 7623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:44.767165 7623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:44.771433 7623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:47:44.771512 7623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:44.771541 7623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:44.771559 7623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:47:44.771574 7623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:47:44.777181 7623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:41559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:41559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:44.778327 7623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:44.779364 7623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:44.788615 7636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:44.788627 7631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:44.788578 7628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:44.788748 7623 server_base.cc:1034] running on GCE node
I20260811 02:47:44.789465 7623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:44.790726 7623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:44.791936 7623 hybrid_clock.cc:648] HybridClock initialized: now 1786416464791906 us; error 50 us; skew 500 ppm
Aug 11 02:47:44 dist-test-slave-n326 krb5kdc[7602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416464, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:44.794998 7623 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:47:44.796034 7623 webserver.cc:460] Webserver started at http://127.21.236.190:38797/ using document root <none> and password file <none>
I20260811 02:47:44.796679 7623 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:44.796787 7623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:44.797052 7623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:44.799469 7623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "c128d2a45a8d4bc1bf5460b3f0006ab5"
format_stamp: "Formatted at 2026-08-11 02:47:44 on dist-test-slave-n326"
server_key: "c629dd67286a166dccaa4e7dc430535e"
server_key_iv: "51b34440a3287b2516baea3113b3f446"
server_key_version: "encryptionkey@0"
I20260811 02:47:44.800104 7623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "c128d2a45a8d4bc1bf5460b3f0006ab5"
format_stamp: "Formatted at 2026-08-11 02:47:44 on dist-test-slave-n326"
server_key: "c629dd67286a166dccaa4e7dc430535e"
server_key_iv: "51b34440a3287b2516baea3113b3f446"
server_key_version: "encryptionkey@0"
I20260811 02:47:44.804266 7623 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:47:44.806929 7648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:44.808365 7623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:47:44.808539 7623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "c128d2a45a8d4bc1bf5460b3f0006ab5"
format_stamp: "Formatted at 2026-08-11 02:47:44 on dist-test-slave-n326"
server_key: "c629dd67286a166dccaa4e7dc430535e"
server_key_iv: "51b34440a3287b2516baea3113b3f446"
server_key_version: "encryptionkey@0"
I20260811 02:47:44.808702 7623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:44.843483 7623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:44.847194 7623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:44.847446 7623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:44.855947 7623 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:41559
I20260811 02:47:44.856021 7740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:41559 every 8 connection(s)
I20260811 02:47:44.857050 7623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:47:44.858830 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 7623
I20260811 02:47:44.858990 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:47:44.859333 22450 external_mini_cluster.cc:1428] Setting key ec03f74d02403c47e6806457ee1a7974
I20260811 02:47:44.860561 7742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:44.867911 7742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5: Bootstrap starting.
Aug 11 02:47:44 dist-test-slave-n326 krb5kdc[7602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:44.870711 7742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:44.871605 7742 log.cc:824] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:44.874078 7742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5: No bootstrap required, opened a new log
I20260811 02:47:44.875268 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:44.860854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41614 (local address 127.21.236.190:41559)
0811 02:47:44.861289 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:44.861305 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:44.861342 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:47:44.862036 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:44.862054 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:44.862381 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:44.862672 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:44.863763 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:44.864822 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:44.866159 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:44.866699 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:44.866845 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:44.869994 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:44.870001 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:44.870014 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:44.870046 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:44.872179 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:44.872753 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:44.872758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:44.872763 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:44.872879 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:44.873193 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:44.873196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:44.873197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:44.873618 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:44.873769 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:44.874117 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 02:47:44.874423 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":160,"threads_started":1}
I20260811 02:47:44.877476 7742 raft_consensus.cc:348] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c128d2a45a8d4bc1bf5460b3f0006ab5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41559 } }
I20260811 02:47:44.877741 7742 raft_consensus.cc:374] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:44.877853 7742 raft_consensus.cc:729] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c128d2a45a8d4bc1bf5460b3f0006ab5, State: Initialized, Role: FOLLOWER
I20260811 02:47:44.878331 7742 consensus_queue.cc:260] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c128d2a45a8d4bc1bf5460b3f0006ab5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41559 } }
I20260811 02:47:44.878476 7742 raft_consensus.cc:388] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:47:44.878571 7742 raft_consensus.cc:482] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:47:44.878684 7742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:44.879769 7742 raft_consensus.cc:504] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c128d2a45a8d4bc1bf5460b3f0006ab5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41559 } }
I20260811 02:47:44.880137 7742 leader_election.cc:302] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c128d2a45a8d4bc1bf5460b3f0006ab5; no voters:
I20260811 02:47:44.880532 7742 leader_election.cc:288] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:47:44.880652 7750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:44.880934 7750 raft_consensus.cc:686] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [term 1 LEADER]: Becoming Leader. State: Replica: c128d2a45a8d4bc1bf5460b3f0006ab5, State: Running, Role: LEADER
I20260811 02:47:44.881284 7750 consensus_queue.cc:237] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c128d2a45a8d4bc1bf5460b3f0006ab5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41559 } }
I20260811 02:47:44.881868 7742 sys_catalog.cc:564] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:47:44.883277 7752 sys_catalog.cc:455] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c128d2a45a8d4bc1bf5460b3f0006ab5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c128d2a45a8d4bc1bf5460b3f0006ab5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41559 } } }
I20260811 02:47:44.883297 7753 sys_catalog.cc:455] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c128d2a45a8d4bc1bf5460b3f0006ab5. Latest consensus state: current_term: 1 leader_uuid: "c128d2a45a8d4bc1bf5460b3f0006ab5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c128d2a45a8d4bc1bf5460b3f0006ab5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 41559 } } }
I20260811 02:47:44.883453 7752 sys_catalog.cc:458] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:44.883453 7753 sys_catalog.cc:458] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:47:44.883936 7763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:47:44.886741 7763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:47:44.892786 7763 catalog_manager.cc:1347] Generated new cluster ID: 8917adfa456447308ac2a4248829d07e
I20260811 02:47:44.892863 7763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:47:44.909744 7763 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:47:44.911226 7763 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:47:44.923509 7763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5: Generated new TSK 0
I20260811 02:47:44.924309 7763 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:47:44.966997 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:41559
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:45.085837 7781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:45.086112 7781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:45.086155 7781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:45.090325 7781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:45.090395 7781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:45.090454 7781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:47:45.095789 7781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:41559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:45.096865 7781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:45.097868 7781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:45.107611 7796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:45.107690 7791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:45.107611 7792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:45.107786 7781 server_base.cc:1034] running on GCE node
I20260811 02:47:45.108352 7781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:45.108992 7781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:45.110183 7781 hybrid_clock.cc:648] HybridClock initialized: now 1786416465110161 us; error 37 us; skew 500 ppm
Aug 11 02:47:45 dist-test-slave-n326 krb5kdc[7602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416465, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:45.117398 7781 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:47:45.118388 7781 webserver.cc:460] Webserver started at http://127.21.236.129:35029/ using document root <none> and password file <none>
I20260811 02:47:45.118935 7781 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:45.118996 7781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:45.119251 7781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:45.121352 7781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1"
format_stamp: "Formatted at 2026-08-11 02:47:45 on dist-test-slave-n326"
server_key: "38a0d9c1be3fbd877a62d33938ef0b52"
server_key_iv: "4c6a691f09e8febc8264c1ad9c8a1946"
server_key_version: "encryptionkey@0"
I20260811 02:47:45.121905 7781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1"
format_stamp: "Formatted at 2026-08-11 02:47:45 on dist-test-slave-n326"
server_key: "38a0d9c1be3fbd877a62d33938ef0b52"
server_key_iv: "4c6a691f09e8febc8264c1ad9c8a1946"
server_key_version: "encryptionkey@0"
I20260811 02:47:45.125929 7781 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:47:45.128515 7809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:45.129772 7781 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:47:45.129923 7781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1"
format_stamp: "Formatted at 2026-08-11 02:47:45 on dist-test-slave-n326"
server_key: "38a0d9c1be3fbd877a62d33938ef0b52"
server_key_iv: "4c6a691f09e8febc8264c1ad9c8a1946"
server_key_version: "encryptionkey@0"
I20260811 02:47:45.130036 7781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:45.180781 7781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:45.184602 7781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:45.184834 7781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:45.185496 7781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:45.186406 7781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:45.186482 7781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:45.186569 7781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:45.186625 7781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:45.196112 7781 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:35369
I20260811 02:47:45.196169 8006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:35369 every 8 connection(s)
I20260811 02:47:45.197089 7781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:47:45.206254 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 7781
I20260811 02:47:45.206390 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:47:45.206800 22450 external_mini_cluster.cc:1428] Setting key 128af3eb941597ad5048f91312c52178
Aug 11 02:47:45 dist-test-slave-n326 krb5kdc[7602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416465, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:45.212410 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.198960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:43369 (local address 127.21.236.190:41559)
0811 02:47:45.199134 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:45.199139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:45.200063 (+ 924us) server_negotiation.cc:407] Connection header received
0811 02:47:45.201466 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:45.201469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:45.201535 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:45.201611 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:45.202796 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.203368 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.204609 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.204839 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.204923 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.207883 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:45.207886 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:45.207889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:45.207912 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:45.209578 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:45.210300 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:45.210303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:45.210304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:45.210402 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:45.210880 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:45.210887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:45.210890 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:45.211242 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:45.211440 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:45.211952 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 02:47:45.212139 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:47:45.213148 8009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.199436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41559 (local address 127.21.236.129:43369)
0811 02:47:45.199903 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:45.199948 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:45.201231 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:45.201776 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:45.201783 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:45.202143 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:45.202665 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.202677 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.203503 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.203506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.204490 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.204501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.204987 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.204990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.205143 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.205765 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:45.205783 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:45.207724 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:45.209710 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:45.209720 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:45.209741 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:45.210164 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:45.210511 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:45.210518 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:45.210524 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:45.210737 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:45.211399 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:45.211404 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:45.211683 (+ 279us) client_negotiation.cc:769] Sending connection context
0811 02:47:45.211863 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 02:47:45.212157 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":120,"threads_started":1}
I20260811 02:47:45.214635 8007 heartbeater.cc:347] Connected to a master server at 127.21.236.190:41559
I20260811 02:47:45.214893 8007 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:45.215497 8007 heartbeater.cc:511] Master 127.21.236.190:41559 requested a full tablet report, sending...
I20260811 02:47:45.217064 7683 ts_manager.cc:194] Registered new tserver with Master: f6bb5e0bf7804e8e832ab21f9b764ee1 (127.21.236.129:35369)
I20260811 02:47:45.218222 7683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:43369
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:47:45.238771 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:41559
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:45.357431 8013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:45.357723 8013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:45.357769 8013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:45.361848 8013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:45.361912 8013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:45.361972 8013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:47:45.367386 8013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:41559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:45.368525 8013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:45.369837 8013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:45.380498 8031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:45.380504 8021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:45.380541 8013 server_base.cc:1034] running on GCE node
W20260811 02:47:45.380569 8028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:45.381229 8013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:45.381922 8013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:45.383041 8013 hybrid_clock.cc:648] HybridClock initialized: now 1786416465383016 us; error 43 us; skew 500 ppm
Aug 11 02:47:45 dist-test-slave-n326 krb5kdc[7602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416465, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:45.386159 8013 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:47:45.387156 8013 webserver.cc:460] Webserver started at http://127.21.236.130:45859/ using document root <none> and password file <none>
I20260811 02:47:45.387753 8013 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:45.387850 8013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:45.388108 8013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:45.390182 8013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "b59151c538784b6890f54df5978163d4"
format_stamp: "Formatted at 2026-08-11 02:47:45 on dist-test-slave-n326"
server_key: "23a0f6ac2f5fc6489eb9c7f02099fd31"
server_key_iv: "f2f56b33562a61039038d93437bcf14e"
server_key_version: "encryptionkey@0"
I20260811 02:47:45.390765 8013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "b59151c538784b6890f54df5978163d4"
format_stamp: "Formatted at 2026-08-11 02:47:45 on dist-test-slave-n326"
server_key: "23a0f6ac2f5fc6489eb9c7f02099fd31"
server_key_iv: "f2f56b33562a61039038d93437bcf14e"
server_key_version: "encryptionkey@0"
I20260811 02:47:45.394670 8013 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:47:45.397524 8042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:45.398823 8013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:47:45.398976 8013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "b59151c538784b6890f54df5978163d4"
format_stamp: "Formatted at 2026-08-11 02:47:45 on dist-test-slave-n326"
server_key: "23a0f6ac2f5fc6489eb9c7f02099fd31"
server_key_iv: "f2f56b33562a61039038d93437bcf14e"
server_key_version: "encryptionkey@0"
I20260811 02:47:45.399130 8013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:45.412878 8013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:45.416821 8013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:45.417068 8013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:45.417728 8013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:45.418653 8013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:45.418730 8013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:45.418808 8013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:45.418855 8013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:45.429150 8013 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:39391
I20260811 02:47:45.429173 8237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:39391 every 8 connection(s)
I20260811 02:47:45.430153 8013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:47:45.436626 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 8013
I20260811 02:47:45.436760 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:47:45.437085 22450 external_mini_cluster.cc:1428] Setting key 098adc860575ec62b493edda0ab3d71b
Aug 11 02:47:45 dist-test-slave-n326 krb5kdc[7602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416465, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:45.445946 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.432228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:57189 (local address 127.21.236.190:41559)
0811 02:47:45.432365 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:45.432368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:45.433300 (+ 932us) server_negotiation.cc:407] Connection header received
0811 02:47:45.435016 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:45.435019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:45.435167 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:45.435258 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:45.436672 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.437388 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.438709 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.438968 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.439143 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.441993 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:45.441997 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:45.442001 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:45.442039 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:45.443644 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:45.444232 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:45.444236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:45.444237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:45.444327 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:45.444747 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:45.444751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:45.444753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:45.444959 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:45.445075 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:45.445626 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 02:47:45.445762 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:47:45.446861 8240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.432590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41559 (local address 127.21.236.130:57189)
0811 02:47:45.433116 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:45.433164 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:45.434750 (+ 1586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:45.435460 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:45.435468 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:45.435856 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:45.436484 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.436507 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.437536 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.437540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.438564 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.438574 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.439182 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.439186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.439398 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.440110 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:45.440128 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:45.441803 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:45.443787 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:45.443794 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:45.443813 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:45.444117 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:45.444481 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:45.444485 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:45.444487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:45.444647 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:45.445081 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:45.445092 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:45.445362 (+ 270us) client_negotiation.cc:769] Sending connection context
0811 02:47:45.445573 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 02:47:45.445935 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
I20260811 02:47:45.448321 8238 heartbeater.cc:347] Connected to a master server at 127.21.236.190:41559
I20260811 02:47:45.448596 8238 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:45.449182 8238 heartbeater.cc:511] Master 127.21.236.190:41559 requested a full tablet report, sending...
I20260811 02:47:45.450349 7683 ts_manager.cc:194] Registered new tserver with Master: b59151c538784b6890f54df5978163d4 (127.21.236.130:39391)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
I20260811 02:47:45.451177 7683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:57189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:47:45.470939 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:41559
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:47:45.587921 8244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:47:45.588213 8244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:47:45.588259 8244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:47:45.592298 8244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:47:45.592358 8244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:47:45.592418 8244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:47:45.597831 8244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:41559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:47:45.598898 8244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:47:45.599964 8244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:47:45.610289 8254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:47:45.610291 8255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:45.610329 8244 server_base.cc:1034] running on GCE node
W20260811 02:47:45.610337 8257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:47:45.610972 8244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:47:45.611655 8244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:47:45.612840 8244 hybrid_clock.cc:648] HybridClock initialized: now 1786416465612801 us; error 48 us; skew 500 ppm
Aug 11 02:47:45 dist-test-slave-n326 krb5kdc[7602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416465, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:47:45.615904 8244 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:47:45.616796 8244 webserver.cc:460] Webserver started at http://127.21.236.131:36283/ using document root <none> and password file <none>
I20260811 02:47:45.617388 8244 fs_manager.cc:359] Metadata directory not provided
I20260811 02:47:45.617482 8244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:47:45.617724 8244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:47:45.619879 8244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "8d22f2ce693f467dad96239130dd0db5"
format_stamp: "Formatted at 2026-08-11 02:47:45 on dist-test-slave-n326"
server_key: "a690f5b2200c4101d4d4d6d9e1750cc7"
server_key_iv: "8d732100ebe5c214a6602dc520560b2e"
server_key_version: "encryptionkey@0"
I20260811 02:47:45.620472 8244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "8d22f2ce693f467dad96239130dd0db5"
format_stamp: "Formatted at 2026-08-11 02:47:45 on dist-test-slave-n326"
server_key: "a690f5b2200c4101d4d4d6d9e1750cc7"
server_key_iv: "8d732100ebe5c214a6602dc520560b2e"
server_key_version: "encryptionkey@0"
I20260811 02:47:45.624390 8244 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:47:45.626979 8269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:45.628312 8244 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:47:45.628468 8244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "8d22f2ce693f467dad96239130dd0db5"
format_stamp: "Formatted at 2026-08-11 02:47:45 on dist-test-slave-n326"
server_key: "a690f5b2200c4101d4d4d6d9e1750cc7"
server_key_iv: "8d732100ebe5c214a6602dc520560b2e"
server_key_version: "encryptionkey@0"
I20260811 02:47:45.628588 8244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:47:45.641752 8244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:47:45.644946 8244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:47:45.645171 8244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:47:45.645772 8244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:47:45.646620 8244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:47:45.646690 8244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:45.646762 8244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:47:45.646813 8244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:47:45.656437 8244 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:44633
I20260811 02:47:45.656468 8466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:44633 every 8 connection(s)
I20260811 02:47:45.657389 8244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:47:45.658972 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 8244
I20260811 02:47:45.659149 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:47:45.659585 22450 external_mini_cluster.cc:1428] Setting key 8cbadf980a266b2bfefefcf3cb5f26ed
Aug 11 02:47:45 dist-test-slave-n326 krb5kdc[7602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416465, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:47:45.672716 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.659537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:55803 (local address 127.21.236.190:41559)
0811 02:47:45.659701 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:45.659706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:45.660675 (+ 969us) server_negotiation.cc:407] Connection header received
0811 02:47:45.662098 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:45.662101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:45.662151 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:45.662224 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:45.663557 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.664412 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.665663 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.665897 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.665993 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.668875 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:45.668878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:45.668880 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:45.668910 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:45.670541 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:45.671099 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:45.671104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:45.671106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:45.671192 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:45.671559 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:45.671563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:45.671565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:45.671760 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:45.671878 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:45.672411 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 02:47:45.672540 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:47:45.673614 8471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.660007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:41559 (local address 127.21.236.131:55803)
0811 02:47:45.660530 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:45.660585 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:45.661903 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:45.662357 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:45.662364 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:45.662718 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:47:45.663343 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.663371 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.664608 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.664610 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.665505 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.665514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.666081 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.666085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.666261 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.666903 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:47:45.666923 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:47:45.668701 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:47:45.670663 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:45.670668 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:45.670680 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:45.670964 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:45.671299 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:47:45.671302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:47:45.671305 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:47:45.671458 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:47:45.671877 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:47:45.671907 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:47:45.672150 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 02:47:45.672344 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 02:47:45.672679 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":159,"threads_started":1}
I20260811 02:47:45.674836 8468 heartbeater.cc:347] Connected to a master server at 127.21.236.190:41559
I20260811 02:47:45.675150 8468 heartbeater.cc:464] Registering TS with master...
I20260811 02:47:45.675653 8468 heartbeater.cc:511] Master 127.21.236.190:41559 requested a full tablet report, sending...
I20260811 02:47:45.676717 7683 ts_manager.cc:194] Registered new tserver with Master: 8d22f2ce693f467dad96239130dd0db5 (127.21.236.131:44633)
I20260811 02:47:45.677294 7683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:55803
I20260811 02:47:45.685877 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:47:45.695672 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.687905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41616 (local address 127.21.236.190:41559)
0811 02:47:45.688080 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:45.688085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:45.688248 (+ 163us) server_negotiation.cc:407] Connection header received
0811 02:47:45.688333 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:45.688335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:45.688379 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:45.688452 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:47:45.689147 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.689729 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.690541 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.690764 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.690846 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.692236 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:47:45.692240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:47:45.692242 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:47:45.692267 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:47:45.693970 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:45.694340 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:45.694343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:45.694344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:45.694462 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:47:45.694699 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:47:45.694701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:47:45.694703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:47:45.694933 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:47:45.695055 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:45.695293 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 02:47:45.695393 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:47:45.698730 7683 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:47:45.701829 7683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:47:45.718130 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.712304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56102 (local address 127.21.236.130:39391)
0811 02:47:45.712768 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:45.712779 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:45.712801 (+ 22us) server_negotiation.cc:407] Connection header received
0811 02:47:45.712875 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:45.712881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:45.713078 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:45.713545 (+ 467us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:45.713734 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.715038 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.716670 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.717307 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.717467 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.717623 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:45.717828 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 02:47:45.717949 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":130,"threads_started":1}
I20260811 02:47:45.718127 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.712112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:39391 (local address 127.0.0.1:56102)
0811 02:47:45.712334 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:45.712348 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:45.712442 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:45.713267 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:45.713276 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:45.713303 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:45.713608 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.713620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.715239 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.715243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.716533 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.716544 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.717458 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.717462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.717591 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.717633 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 02:47:45.717778 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 02:47:45.717949 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":72,"spinlock_wait_cycles":2432,"thread_start_us":99,"threads_started":1}
I20260811 02:47:45.718632 8487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.711605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52536 (local address 127.21.236.131:44633)
0811 02:47:45.713260 (+ 1655us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:45.713273 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:45.713296 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:47:45.713402 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:45.713411 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:45.713802 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:45.713975 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:45.714173 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.714896 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.716735 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.717573 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.717694 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.717811 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:45.718382 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 02:47:45.718480 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1549,"thread_start_us":155,"threads_started":1}
I20260811 02:47:45.718884 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.711508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53340 (local address 127.21.236.129:35369)
0811 02:47:45.711829 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:45.711848 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:45.712534 (+ 686us) server_negotiation.cc:407] Connection header received
0811 02:47:45.712663 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:45.712669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:45.712904 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:45.713097 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:45.713722 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.715030 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.716753 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.717573 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.717694 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.717811 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:45.718637 (+ 826us) server_negotiation.cc:299] Negotiation successful
0811 02:47:45.718742 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":92,"threads_started":1}
I20260811 02:47:45.719055 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.711240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:44633 (local address 127.0.0.1:52536)
0811 02:47:45.712795 (+ 1555us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:45.712807 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:45.712908 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:45.713963 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:45.713966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:45.713985 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:45.714078 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.714085 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.715027 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.715032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.716581 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.716593 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.717879 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.717895 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.718136 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.718222 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 02:47:45.718289 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:47:45.718823 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1490,"mutex_wait_us":1896,"thread_start_us":123,"threads_started":1}
I20260811 02:47:45.718530 8485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.711240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:35369 (local address 127.0.0.1:53340)
0811 02:47:45.712124 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:45.712193 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:45.712347 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:45.713297 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:45.713300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:45.713313 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:45.713553 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.713563 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.715174 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.715177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.716622 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.716632 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.717879 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.717883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.718134 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.718217 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 02:47:45.718289 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 02:47:45.718395 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":430,"thread_start_us":156,"threads_started":1}
I20260811 02:47:45.722117 8122 tablet_service.cc:1467] Processing CreateTablet for tablet 1b1957c5bfbb458e9bae342a855b38df (DEFAULT_TABLE table=test-table [id=9d9504f5a452486bb84b097bbc4d3136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:45.722657 8352 tablet_service.cc:1467] Processing CreateTablet for tablet 1b1957c5bfbb458e9bae342a855b38df (DEFAULT_TABLE table=test-table [id=9d9504f5a452486bb84b097bbc4d3136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:45.723479 8122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b1957c5bfbb458e9bae342a855b38df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:45.723192 7899 tablet_service.cc:1467] Processing CreateTablet for tablet 1b1957c5bfbb458e9bae342a855b38df (DEFAULT_TABLE table=test-table [id=9d9504f5a452486bb84b097bbc4d3136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:47:45.723820 8352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b1957c5bfbb458e9bae342a855b38df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:45.724215 7899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b1957c5bfbb458e9bae342a855b38df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:47:45.729722 8491 tablet_bootstrap.cc:492] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4: Bootstrap starting.
I20260811 02:47:45.731361 8494 tablet_bootstrap.cc:492] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5: Bootstrap starting.
I20260811 02:47:45.732538 8499 tablet_bootstrap.cc:492] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1: Bootstrap starting.
I20260811 02:47:45.733278 8491 tablet_bootstrap.cc:654] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:45.734007 8494 tablet_bootstrap.cc:654] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:45.734294 8491 log.cc:824] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:45.734858 8494 log.cc:824] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:45.734939 8499 tablet_bootstrap.cc:654] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1: Neither blocks nor log segments found. Creating new log.
I20260811 02:47:45.735915 8499 log.cc:824] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1: Log is configured to *not* fsync() on all Append() calls
I20260811 02:47:45.736536 8491 tablet_bootstrap.cc:492] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4: No bootstrap required, opened a new log
I20260811 02:47:45.736748 8491 ts_tablet_manager.cc:1397] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 02:47:45.736868 8494 tablet_bootstrap.cc:492] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5: No bootstrap required, opened a new log
I20260811 02:47:45.737047 8494 ts_tablet_manager.cc:1397] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:47:45.737867 8499 tablet_bootstrap.cc:492] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1: No bootstrap required, opened a new log
I20260811 02:47:45.738073 8499 ts_tablet_manager.cc:1397] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:47:45.740855 8491 raft_consensus.cc:348] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 35369 } } peers { permanent_uuid: "8d22f2ce693f467dad96239130dd0db5" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44633 } } peers { permanent_uuid: "b59151c538784b6890f54df5978163d4" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 39391 } }
I20260811 02:47:45.740855 8494 raft_consensus.cc:348] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 35369 } } peers { permanent_uuid: "8d22f2ce693f467dad96239130dd0db5" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44633 } } peers { permanent_uuid: "b59151c538784b6890f54df5978163d4" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 39391 } }
I20260811 02:47:45.741147 8494 raft_consensus.cc:374] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:45.741147 8491 raft_consensus.cc:374] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:45.741225 8491 raft_consensus.cc:729] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b59151c538784b6890f54df5978163d4, State: Initialized, Role: FOLLOWER
I20260811 02:47:45.741223 8494 raft_consensus.cc:729] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d22f2ce693f467dad96239130dd0db5, State: Initialized, Role: FOLLOWER
I20260811 02:47:45.741752 8494 consensus_queue.cc:260] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 35369 } } peers { permanent_uuid: "8d22f2ce693f467dad96239130dd0db5" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44633 } } peers { permanent_uuid: "b59151c538784b6890f54df5978163d4" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 39391 } }
I20260811 02:47:45.741752 8491 consensus_queue.cc:260] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 35369 } } peers { permanent_uuid: "8d22f2ce693f467dad96239130dd0db5" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44633 } } peers { permanent_uuid: "b59151c538784b6890f54df5978163d4" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 39391 } }
I20260811 02:47:45.741950 8499 raft_consensus.cc:348] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 35369 } } peers { permanent_uuid: "8d22f2ce693f467dad96239130dd0db5" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44633 } } peers { permanent_uuid: "b59151c538784b6890f54df5978163d4" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 39391 } }
I20260811 02:47:45.742244 8499 raft_consensus.cc:374] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:47:45.742321 8499 raft_consensus.cc:729] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6bb5e0bf7804e8e832ab21f9b764ee1, State: Initialized, Role: FOLLOWER
I20260811 02:47:45.742762 8238 heartbeater.cc:503] Master 127.21.236.190:41559 was elected leader, sending a full tablet report...
I20260811 02:47:45.742813 8494 ts_tablet_manager.cc:1428] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:47:45.742923 8468 heartbeater.cc:503] Master 127.21.236.190:41559 was elected leader, sending a full tablet report...
I20260811 02:47:45.743852 8491 ts_tablet_manager.cc:1428] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 02:47:45.743845 8499 consensus_queue.cc:260] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 35369 } } peers { permanent_uuid: "8d22f2ce693f467dad96239130dd0db5" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44633 } } peers { permanent_uuid: "b59151c538784b6890f54df5978163d4" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 39391 } }
I20260811 02:47:45.744588 8007 heartbeater.cc:503] Master 127.21.236.190:41559 was elected leader, sending a full tablet report...
I20260811 02:47:45.744946 8499 ts_tablet_manager.cc:1428] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 02:47:45.765141 8508 raft_consensus.cc:482] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:47:45.765413 8508 raft_consensus.cc:504] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 35369 } } peers { permanent_uuid: "8d22f2ce693f467dad96239130dd0db5" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44633 } } peers { permanent_uuid: "b59151c538784b6890f54df5978163d4" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 39391 } }
I20260811 02:47:45.766707 8508 leader_election.cc:288] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d22f2ce693f467dad96239130dd0db5 (127.21.236.131:44633), b59151c538784b6890f54df5978163d4 (127.21.236.130:39391)
I20260811 02:47:45.772547 8487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.767219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:44675 (local address 127.21.236.131:44633)
0811 02:47:45.767425 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:45.767429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:45.767574 (+ 145us) server_negotiation.cc:407] Connection header received
0811 02:47:45.767703 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:45.767707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:45.767767 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:45.767862 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:45.768304 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.769108 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.771036 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.771703 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.771845 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.771901 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:45.772319 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 02:47:45.772448 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260811 02:47:45.772535 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.767154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:49327 (local address 127.21.236.130:39391)
0811 02:47:45.767307 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:45.767313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:45.767676 (+ 363us) server_negotiation.cc:407] Connection header received
0811 02:47:45.767747 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:45.767752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:45.767810 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:45.767889 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:47:45.768310 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.769109 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.771036 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.771704 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.771845 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.771902 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:45.772319 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 02:47:45.772402 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:47:45.772634 8511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.766988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:39391 (local address 127.21.236.129:49327)
0811 02:47:45.767559 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:45.767570 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:45.767660 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:45.767904 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:45.767907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:45.767950 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:45.768165 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.768173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.769247 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.769251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.770842 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.770848 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.771853 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.771857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.772130 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.772196 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 02:47:45.772275 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 02:47:45.772396 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":379,"spinlock_wait_cycles":8576,"thread_start_us":143,"threads_started":1}
I20260811 02:47:45.772643 8509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.766988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:44633 (local address 127.21.236.129:44675)
0811 02:47:45.767429 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:45.767445 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:47:45.767596 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:47:45.767904 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:47:45.767907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:47:45.767949 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:47:45.768165 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.768173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.769247 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.769251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.770835 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:47:45.770849 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.771853 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.771857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:47:45.772111 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.772188 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 02:47:45.772275 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 02:47:45.772396 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":328,"spinlock_wait_cycles":28288,"thread_start_us":163,"threads_started":1}
I20260811 02:47:45.773330 8389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b1957c5bfbb458e9bae342a855b38df" candidate_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d22f2ce693f467dad96239130dd0db5" is_pre_election: true
I20260811 02:47:45.773322 8155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b1957c5bfbb458e9bae342a855b38df" candidate_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b59151c538784b6890f54df5978163d4" is_pre_election: true
I20260811 02:47:45.773687 8389 raft_consensus.cc:2454] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb5e0bf7804e8e832ab21f9b764ee1 in term 0.
I20260811 02:47:45.773687 8155 raft_consensus.cc:2454] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6bb5e0bf7804e8e832ab21f9b764ee1 in term 0.
I20260811 02:47:45.774251 7813 leader_election.cc:302] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8d22f2ce693f467dad96239130dd0db5, b59151c538784b6890f54df5978163d4, f6bb5e0bf7804e8e832ab21f9b764ee1; no voters:
I20260811 02:47:45.774595 8508 raft_consensus.cc:2781] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:47:45.774700 8508 raft_consensus.cc:482] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:47:45.774767 8508 raft_consensus.cc:3037] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:45.775974 8508 raft_consensus.cc:504] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 35369 } } peers { permanent_uuid: "8d22f2ce693f467dad96239130dd0db5" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44633 } } peers { permanent_uuid: "b59151c538784b6890f54df5978163d4" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 39391 } }
I20260811 02:47:45.776397 8508 leader_election.cc:288] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [CANDIDATE]: Term 1 election: Requested vote from peers 8d22f2ce693f467dad96239130dd0db5 (127.21.236.131:44633), b59151c538784b6890f54df5978163d4 (127.21.236.130:39391)
I20260811 02:47:45.776803 8389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b1957c5bfbb458e9bae342a855b38df" candidate_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d22f2ce693f467dad96239130dd0db5"
I20260811 02:47:45.776862 8155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b1957c5bfbb458e9bae342a855b38df" candidate_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b59151c538784b6890f54df5978163d4"
I20260811 02:47:45.776949 8389 raft_consensus.cc:3037] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:45.777006 8155 raft_consensus.cc:3037] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:47:45.778229 8389 raft_consensus.cc:2454] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bb5e0bf7804e8e832ab21f9b764ee1 in term 1.
I20260811 02:47:45.778491 8155 raft_consensus.cc:2454] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6bb5e0bf7804e8e832ab21f9b764ee1 in term 1.
I20260811 02:47:45.778548 7812 leader_election.cc:302] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d22f2ce693f467dad96239130dd0db5, f6bb5e0bf7804e8e832ab21f9b764ee1; no voters:
I20260811 02:47:45.778764 8508 raft_consensus.cc:2781] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:47:45.779043 8508 raft_consensus.cc:686] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [term 1 LEADER]: Becoming Leader. State: Replica: f6bb5e0bf7804e8e832ab21f9b764ee1, State: Running, Role: LEADER
I20260811 02:47:45.779464 8508 consensus_queue.cc:237] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 35369 } } peers { permanent_uuid: "8d22f2ce693f467dad96239130dd0db5" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44633 } } peers { permanent_uuid: "b59151c538784b6890f54df5978163d4" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 39391 } }
I20260811 02:47:45.783272 7674 catalog_manager.cc:5495] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6bb5e0bf7804e8e832ab21f9b764ee1 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6bb5e0bf7804e8e832ab21f9b764ee1" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 35369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d22f2ce693f467dad96239130dd0db5" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b59151c538784b6890f54df5978163d4" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 39391 } health_report { overall_health: UNKNOWN } } }
I20260811 02:47:45.801910 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.798171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53348 (local address 127.21.236.129:35369)
0811 02:47:45.798338 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 02:47:45.798342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:47:45.798357 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:47:45.798445 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:47:45.798447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:47:45.798506 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:47:45.798610 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:47:45.798891 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.799594 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.800454 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:47:45.800684 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:47:45.800777 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:47:45.801100 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:47:45.801570 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:47:45.801672 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 02:47:45.801731 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 02:47:45.801786 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 02:47:45.808329 8389 raft_consensus.cc:1261] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5 [term 1 FOLLOWER]: Refusing update from remote peer f6bb5e0bf7804e8e832ab21f9b764ee1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:45.808571 8155 raft_consensus.cc:1261] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4 [term 1 FOLLOWER]: Refusing update from remote peer f6bb5e0bf7804e8e832ab21f9b764ee1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:47:45.809188 8513 consensus_queue.cc:1035] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d22f2ce693f467dad96239130dd0db5" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:45.809347 8508 consensus_queue.cc:1035] T 1b1957c5bfbb458e9bae342a855b38df P f6bb5e0bf7804e8e832ab21f9b764ee1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b59151c538784b6890f54df5978163d4" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:47:45.820966 8519 mvcc.cc:205] Tried to move back new op lower bound from 7317161843943448576 to 7317161843835703296. Current Snapshot: MvccSnapshot[applied={T|T < 7317161843943448576 or (T in {7317161843943448576})}]
I20260811 02:47:45.826679 7674 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:45.826886 7674 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:47:45.829632 7674 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c128d2a45a8d4bc1bf5460b3f0006ab5: Sending DeleteTablet for 3 replicas of tablet 1b1957c5bfbb458e9bae342a855b38df
I20260811 02:47:45.830628 8352 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b1957c5bfbb458e9bae342a855b38df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:45 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:52536
I20260811 02:47:45.830653 8122 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b1957c5bfbb458e9bae342a855b38df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:45 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:56102
I20260811 02:47:45.830780 7899 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b1957c5bfbb458e9bae342a855b38df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:47:45 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:53340
I20260811 02:47:45.831211 8533 tablet_replica.cc:330] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5: stopping tablet replica
I20260811 02:47:45.831614 8533 raft_consensus.cc:2229] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:45.831763 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 7781
I20260811 02:47:45.855980 8533 raft_consensus.cc:2258] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:45.858292 8533 ts_tablet_manager.cc:1905] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:47:45.862164 8533 ts_tablet_manager.cc:1918] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:47:45.862296 8533 log.cc:1196] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/1b1957c5bfbb458e9bae342a855b38df
I20260811 02:47:45.862675 8533 ts_tablet_manager.cc:1939] T 1b1957c5bfbb458e9bae342a855b38df P 8d22f2ce693f467dad96239130dd0db5: Deleting consensus metadata
I20260811 02:47:45.862690 8534 tablet_replica.cc:330] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4: stopping tablet replica
I20260811 02:47:45.863283 8534 raft_consensus.cc:2229] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:47:45.863720 8534 raft_consensus.cc:2258] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:47:45.864193 7653 catalog_manager.cc:4841] TS 8d22f2ce693f467dad96239130dd0db5 (127.21.236.131:44633): tablet 1b1957c5bfbb458e9bae342a855b38df (table test-table [id=9d9504f5a452486bb84b097bbc4d3136]) successfully deleted
I20260811 02:47:45.865556 8534 ts_tablet_manager.cc:1905] T 1b1957c5bfbb458e9bae342a855b38df P b59151c538784b6890f54df5978163d4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 02:47:45.867684 7655 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:35369 (error 108)
I20260811 02:47:45.867980 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 8013
I20260811 02:47:45.880659 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.880295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:35369 (local address 127.0.0.1:53352)
0811 02:47:45.880479 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:45.880571 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:35369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 02:47:45.880921 7655 catalog_manager.cc:4584] TS f6bb5e0bf7804e8e832ab21f9b764ee1 (127.21.236.129:35369): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1b1957c5bfbb458e9bae342a855b38df: Network error: Client connection negotiation failed: client connection to 127.21.236.129:35369: connect: Connection refused (error 111)
I20260811 02:47:45.887161 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 8244
I20260811 02:47:45.906587 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:47:45.887434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:39391 (local address 127.0.0.1:56112)
0811 02:47:45.906405 (+ 18971us) negotiation.cc:107] Waiting for socket to connect
0811 02:47:45.906495 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.130:39391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18847}
I20260811 02:47:45.913033 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 7623
2026-08-11T02:47:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2733 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:47:45 dist-test-slave-n326 krb5kdc[8539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:47:45 dist-test-slave-n326 krb5kdc[8539](info): set up 2 sockets
Aug 11 02:47:45 dist-test-slave-n326 krb5kdc[8539](info): commencing operation
krb5kdc: starting...
W20260811 02:47:47.296873 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.355s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:47:47 dist-test-slave-n326 krb5kdc[8539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:47:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:47:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/rangeradmin_127.21.236.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/rangeradmin_127.21.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/rangerlookup_127.21.236.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/rangerlookup_127.21.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/HTTP_127.21.236.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/HTTP_127.21.236.148@KRBTEST.COM.keytab.
I20260811 02:47:47.489116 22450 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskQxaM5w/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/postgres -l logfile start
2026-08-11 02:47:50.244 UTC [8593] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 02:47:50.244 UTC [8593] LOG: listening on IPv4 address "127.21.236.148", port 58179
2026-08-11 02:47:50.249 UTC [8593] LOG: listening on Unix socket "/tmp/.s.PGSQL.58179"
2026-08-11 02:47:50.273 UTC [8602] LOG: database system was shut down at 2026-08-11 02:47:48 UTC
2026-08-11 02:47:50.277 UTC [8593] LOG: database system is ready to accept connections
I20260811 02:47:51.492785 22450 mini_postgres.cc:96] Postgres bound to 58179
2026-08-11 02:47:51.500 UTC [8616] FATAL: database "slave" does not exist
127.21.236.148:58179 - accepting connections
I20260811 02:47:51.501489 22450 mini_ranger.cc:161] Starting Ranger...
I20260811 02:47:51.518765 22450 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 02:47:52.073112 22450 mini_ranger.cc:87] Created ranger Postgres database
I20260811 02:47:52.073232 22450 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-admin
2026-08-11 02:47:52,389 [I] DB FLAVOR :POSTGRES
2026-08-11 02:47:52,389 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 02:47:52,389 [I] Checking connection..
2026-08-11 02:47:52,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 02:47:52,639 [I] Checking connection passed.
2026-08-11 02:47:52,639 [I] --------- Verifying version history table ---------
2026-08-11 02:47:52,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:47:52,885 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 02:47:52,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:47:53,133 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 02:47:53,134 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 02:47:53,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 02:47:53.404 UTC [8743] WARNING: there is no transaction in progress
2026-08-11 02:47:53,417 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 02:47:53,417 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:47:53,663 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 02:47:53,663 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 02:47:53,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 02:47:53,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 02:47:54,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-n326.c.gcp-upstream.internal','N') ;"
2026-08-11 02:47:54,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 02:47:54,769 [I] Table x_portal_user does not exist in database ranger
2026-08-11 02:47:54,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 02:47:55,040 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 02:47:55,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 02:47:55,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 02:47:55,552 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 02:47:55,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 02:47:56.571 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.589 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.607 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.623 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.640 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.659 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.717 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.725 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.740 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.751 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.761 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.775 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.786 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.793 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.801 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:56.808 UTC [8939] WARNING: there is no transaction in progress
2026-08-11 02:47:57,531 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 02:47:57,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 02:47:57,781 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 02:47:57,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 02:47:58,058 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 02:47:58,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 02:47:58,331 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 02:47:58,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 02:47:58,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 02:47:58,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-n326.c.gcp-upstream.internal';"
2026-08-11 02:47:59,094 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 02:47:59,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 02:47:59,362 [I] DB_PATCHES have already been applied
I20260811 02:47:59.368393 22450 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-admin:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskQxaM5w/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 02:47:59.368465 22450 mini_ranger.cc:193] Using host: 127.21.236.148
I20260811 02:47:59.373363 22450 mini_ranger.cc:227] Ranger admin URL: http://127.21.236.148:58887
Aug 11, 2026 2:47:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:47:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:47:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 2:47:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 2:47:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 2:48:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 2:48:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 02:48:00.810035 22450 mini_ranger.cc:160] Time spent starting Ranger: real 9.309s user 0.003s sys 0.011s
Aug 11, 2026 2:48:21 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 2:48:21 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 2:48:21 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 2:48:21 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 2:48:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 2:48:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 2:48:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 2:48:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 2:48:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 2:48:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 2:48:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 2:48:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 2:48:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 2:48:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 2:48:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 2:48:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 2:48:21 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 2:48:22 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 02:48:23.011237 22450 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/rangerkms_127.21.236.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/rangerkms_127.21.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.236.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.236.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.236.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/HTTP_127.21.236.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.236.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/HTTP_127.21.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 02:48:23 dist-test-slave-n326 krb5kdc[8539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416503, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 02:48:23.102257 22450 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 02:48:23.113476 22450 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 02:48:24.027518 22450 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 02:48:24.027606 22450 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-kms
I20260811 02:48:24.027704 22450 mini_ranger_kms.cc:227] Using postgres at 127.21.236.148:58179
2026-08-11 02:48:24,768 [I] DB FLAVOR :POSTGRES
2026-08-11 02:48:24,769 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 02:48:24,769 [I] Checking connection
2026-08-11 02:48:24,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 02:48:25,056 [I] connection success
2026-08-11 02:48:25,056 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 02:48:25,056 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 02:48:25,056 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 02:48:25,302 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 02:48:25,302 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 02:48:25,302 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 02:48:25,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:58179/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 02:48:25,687 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 02:48:26.183755 22450 mini_ranger_kms.cc:310] Created kms service
I20260811 02:48:26.423738 22450 mini_ranger_kms.cc:326] Created kudu user
I20260811 02:48:26.471998 22450 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 02:48:26 dist-test-slave-n326 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416503, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:48:26.887593 22450 mini_ranger_kms.cc:384] Added ranger policy
I20260811 02:48:26.887765 22450 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-kms:/tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 02:48:26.887792 22450 mini_ranger_kms.cc:240] Using host: 127.21.236.148
I20260811 02:48:26.892483 22450 mini_ranger_kms.cc:276] Ranger KMS PID: 9235
I20260811 02:48:26.892647 22450 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.236.148:47101
Aug 11, 2026 2:48:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:48:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:48:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 2:48:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 2:48:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 2:48:28 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 02:48:29 dist-test-slave-n326 krb5kdc[8539](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786416509, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.236.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 02:48:29 dist-test-slave-n326 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416509, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.236.148@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:48:29.835214 22450 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.733s user 0.028s sys 0.275s
I20260811 02:48:29.835350 22450 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 02:48:29.835453 22450 mini_ranger_kms.cc:401] 127.21.236.148:47101/kms/v1/keys
Aug 11 02:48:30 dist-test-slave-n326 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416509, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.236.148@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
Aug 11 02:48:32 dist-test-slave-n326 krb5kdc[8539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:48:32.476759 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:36883
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:36883
--ranger_config_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.236.148:47101/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:48:32.602730 9327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:48:32.603015 9327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:48:32.603055 9327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:48:32.607143 9327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:48:32.607208 9327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:48:32.607227 9327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:48:32.607244 9327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:48:32.607259 9327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:48:32.612739 9327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.236.148:47101/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:36883
--ranger_config_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:36883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:48:32.613833 9327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:48:32.614902 9327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:48:32.624928 9327 server_base.cc:1034] running on GCE node
W20260811 02:48:32.624958 9341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:48:32.625121 9336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:48:32.625250 9337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:48:32.625917 9327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:48:32.627146 9327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:48:32.628346 9327 hybrid_clock.cc:648] HybridClock initialized: now 1786416512628301 us; error 36 us; skew 500 ppm
Aug 11 02:48:32 dist-test-slave-n326 krb5kdc[8539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416512, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:48:32.635987 9327 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:48:32.637053 9327 webserver.cc:460] Webserver started at http://127.21.236.190:43207/ using document root <none> and password file <none>
I20260811 02:48:32.637692 9327 fs_manager.cc:359] Metadata directory not provided
I20260811 02:48:32.637785 9327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:48:32.638047 9327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:48:32 dist-test-slave-n326 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416512, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:48:32.705844 9327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "91b08c5f96f84777bb81f65a2f7c9e28"
format_stamp: "Formatted at 2026-08-11 02:48:32 on dist-test-slave-n326"
server_key: "91ba1544447b08329e0bd585182e0850"
server_key_iv: "2c6c8afeb7a5cca0cb8b46280f10fc25"
server_key_version: "kuduclusterkey@0"
I20260811 02:48:32.706586 9327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "91b08c5f96f84777bb81f65a2f7c9e28"
format_stamp: "Formatted at 2026-08-11 02:48:32 on dist-test-slave-n326"
server_key: "91ba1544447b08329e0bd585182e0850"
server_key_iv: "2c6c8afeb7a5cca0cb8b46280f10fc25"
server_key_version: "kuduclusterkey@0"
I20260811 02:48:32.711836 9327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 02:48:32.726888 9359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:48:32.728468 9327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:48:32.728641 9327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "91b08c5f96f84777bb81f65a2f7c9e28"
format_stamp: "Formatted at 2026-08-11 02:48:32 on dist-test-slave-n326"
server_key: "91ba1544447b08329e0bd585182e0850"
server_key_iv: "2c6c8afeb7a5cca0cb8b46280f10fc25"
server_key_version: "kuduclusterkey@0"
I20260811 02:48:32.728798 9327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:48:32.741822 9327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:48:32.744805 9327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:48:32.745023 9327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:48:32.753054 9327 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:36883
I20260811 02:48:32.753084 9450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:36883 every 8 connection(s)
I20260811 02:48:32.754060 9327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:48:32.755990 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 9327
I20260811 02:48:32.756138 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:48:32.756488 22450 external_mini_cluster.cc:1428] Setting key bb903f6e6e512218b421ffaf3204227a
I20260811 02:48:32.757604 9453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:48:32.763504 9453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28: Bootstrap starting.
Aug 11 02:48:32 dist-test-slave-n326 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:48:32.766151 9453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28: Neither blocks nor log segments found. Creating new log.
I20260811 02:48:32.767148 9453 log.cc:824] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28: Log is configured to *not* fsync() on all Append() calls
I20260811 02:48:32.769420 9453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28: No bootstrap required, opened a new log
I20260811 02:48:32.772289 9460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:32.758014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52298 (local address 127.21.236.190:36883)
0811 02:48:32.758507 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 02:48:32.758523 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:48:32.758559 (+ 36us) server_negotiation.cc:407] Connection header received
0811 02:48:32.759451 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:48:32.759470 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:48:32.759794 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:48:32.760138 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:48:32.761124 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:32.762290 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:32.763220 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:32.763762 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:32.763936 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:32.766883 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:48:32.766889 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:48:32.766903 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:48:32.766933 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:48:32.769375 (+ 2442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:48:32.769896 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:48:32.769900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:48:32.769905 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:48:32.770030 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:48:32.770335 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:48:32.770337 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:48:32.770338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:48:32.770730 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:48:32.770888 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 02:48:32.771250 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 02:48:32.771545 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":158,"threads_started":1}
I20260811 02:48:32.773078 9453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b08c5f96f84777bb81f65a2f7c9e28" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36883 } }
I20260811 02:48:32.773339 9453 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:48:32.773414 9453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b08c5f96f84777bb81f65a2f7c9e28, State: Initialized, Role: FOLLOWER
I20260811 02:48:32.773846 9453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b08c5f96f84777bb81f65a2f7c9e28" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36883 } }
I20260811 02:48:32.774008 9453 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:48:32.774086 9453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:48:32.774215 9453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:48:32.775198 9453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b08c5f96f84777bb81f65a2f7c9e28" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36883 } }
I20260811 02:48:32.775537 9453 leader_election.cc:302] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91b08c5f96f84777bb81f65a2f7c9e28; no voters:
I20260811 02:48:32.775906 9453 leader_election.cc:288] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:48:32.775985 9462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:48:32.776211 9462 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [term 1 LEADER]: Becoming Leader. State: Replica: 91b08c5f96f84777bb81f65a2f7c9e28, State: Running, Role: LEADER
I20260811 02:48:32.776520 9462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b08c5f96f84777bb81f65a2f7c9e28" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36883 } }
I20260811 02:48:32.777249 9453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:48:32.778560 9465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91b08c5f96f84777bb81f65a2f7c9e28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b08c5f96f84777bb81f65a2f7c9e28" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36883 } } }
I20260811 02:48:32.778532 9464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91b08c5f96f84777bb81f65a2f7c9e28. Latest consensus state: current_term: 1 leader_uuid: "91b08c5f96f84777bb81f65a2f7c9e28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91b08c5f96f84777bb81f65a2f7c9e28" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36883 } } }
I20260811 02:48:32.778700 9465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [sys.catalog]: This master's current role is: LEADER
I20260811 02:48:32.778787 9464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28 [sys.catalog]: This master's current role is: LEADER
I20260811 02:48:32.779229 9479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:48:32.782836 9479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:48:32.783421 9453 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 02:48:32.794436 9479 catalog_manager.cc:1347] Generated new cluster ID: de36354dd6934318a63156e375cfc882
I20260811 02:48:32.794538 9479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:48:32.817102 9479 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:48:32.818262 9479 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:48:32.826483 9479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28: Generated new TSK 0
I20260811 02:48:32.827342 9479 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:48:33.610996 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.236.148:47101/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:36883
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:48:33.769198 9530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:48:33.769622 9530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:48:33.769738 9530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:48:33.775843 9530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:48:33.775974 9530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:48:33.776109 9530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:48:33.783962 9530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.236.148:47101/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:36883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:48:33.785308 9530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:48:33.786599 9530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:48:33.798787 9541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:48:33.798985 9547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:48:33.799320 9530 server_base.cc:1034] running on GCE node
W20260811 02:48:33.799093 9540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:48:33.800040 9530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:48:33.800841 9530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:48:33.802044 9530 hybrid_clock.cc:648] HybridClock initialized: now 1786416513802021 us; error 35 us; skew 500 ppm
Aug 11 02:48:33 dist-test-slave-n326 krb5kdc[8539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416513, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:48:33.805794 9530 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:48:33.806849 9530 webserver.cc:460] Webserver started at http://127.21.236.129:40831/ using document root <none> and password file <none>
I20260811 02:48:33.807601 9530 fs_manager.cc:359] Metadata directory not provided
I20260811 02:48:33.807693 9530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:48:33.807934 9530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:48:33 dist-test-slave-n326 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416513, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:48:33.825742 9530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "bbb88e805e724ebd867029523e0937e3"
format_stamp: "Formatted at 2026-08-11 02:48:33 on dist-test-slave-n326"
server_key: "ffb503583665a100774fd587dd0d30af"
server_key_iv: "b1a944e3eaeafea3684633ccd4a4198f"
server_key_version: "kuduclusterkey@0"
I20260811 02:48:33.826545 9530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "bbb88e805e724ebd867029523e0937e3"
format_stamp: "Formatted at 2026-08-11 02:48:33 on dist-test-slave-n326"
server_key: "ffb503583665a100774fd587dd0d30af"
server_key_iv: "b1a944e3eaeafea3684633ccd4a4198f"
server_key_version: "kuduclusterkey@0"
I20260811 02:48:33.832715 9530 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260811 02:48:33.859539 9567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:48:33.875777 9530 fs_manager.cc:725] Time spent opening block manager: real 0.026s user 0.003s sys 0.001s
I20260811 02:48:33.875984 9530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "bbb88e805e724ebd867029523e0937e3"
format_stamp: "Formatted at 2026-08-11 02:48:33 on dist-test-slave-n326"
server_key: "ffb503583665a100774fd587dd0d30af"
server_key_iv: "b1a944e3eaeafea3684633ccd4a4198f"
server_key_version: "kuduclusterkey@0"
I20260811 02:48:33.876171 9530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:48:33.913815 9530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:48:33.918030 9530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:48:33.918303 9530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:48:33.919090 9530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:48:33.920261 9530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:48:33.920382 9530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:48:33.920524 9530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:48:33.920630 9530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:48:33.939816 9530 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:45355
I20260811 02:48:33.941291 9530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:48:33.942408 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 9530
I20260811 02:48:33.942569 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:48:33.942936 22450 external_mini_cluster.cc:1428] Setting key d59f29721c4f8b2a5d65ffadf7271a85
I20260811 02:48:33.953847 9793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:45355 every 8 connection(s)
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
Aug 11 02:48:33 dist-test-slave-n326 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416513, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:48:33.986037 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:33.953855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:35813 (local address 127.21.236.190:36883)
0811 02:48:33.965824 (+ 11969us) server_negotiation.cc:206] Beginning negotiation
0811 02:48:33.965830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:48:33.965847 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:48:33.965898 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:48:33.965904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:48:33.965987 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:48:33.966099 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:48:33.968022 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:33.968681 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:33.970775 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:33.971239 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:33.971323 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:33.977514 (+ 6191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:48:33.977519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:48:33.977522 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:48:33.977556 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:48:33.979785 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:48:33.983522 (+ 3737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:48:33.983527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:48:33.983529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:48:33.983648 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:48:33.984171 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:48:33.984176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:48:33.984180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:48:33.984509 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:48:33.984639 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 02:48:33.985534 (+ 895us) server_negotiation.cc:299] Negotiation successful
0811 02:48:33.985716 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11860,"thread_start_us":90,"threads_started":1}
I20260811 02:48:33.988094 9798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:33.952448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:36883 (local address 127.21.236.129:35813)
0811 02:48:33.963677 (+ 11229us) negotiation.cc:107] Waiting for socket to connect
0811 02:48:33.963779 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0811 02:48:33.965266 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:48:33.966500 (+ 1234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:48:33.966512 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:48:33.966945 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:48:33.967790 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:33.967817 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:33.969000 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:33.969004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:33.970619 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:33.970631 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:33.974187 (+ 3556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:33.974191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:33.974442 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:33.975285 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:48:33.975310 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:48:33.977258 (+ 1948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:48:33.979956 (+ 2698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:48:33.979963 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:48:33.979984 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:48:33.980376 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:48:33.983783 (+ 3407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:48:33.983786 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:48:33.983788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:48:33.984003 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:48:33.984715 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:48:33.984727 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:48:33.985161 (+ 434us) client_negotiation.cc:769] Sending connection context
0811 02:48:33.986327 (+ 1166us) client_negotiation.cc:240] Negotiation successful
0811 02:48:33.986770 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10695,"thread_start_us":236,"threads_started":1}
I20260811 02:48:33.989900 9794 heartbeater.cc:347] Connected to a master server at 127.21.236.190:36883
I20260811 02:48:33.990191 9794 heartbeater.cc:464] Registering TS with master...
I20260811 02:48:33.990880 9794 heartbeater.cc:511] Master 127.21.236.190:36883 requested a full tablet report, sending...
I20260811 02:48:33.993052 9392 ts_manager.cc:194] Registered new tserver with Master: bbb88e805e724ebd867029523e0937e3 (127.21.236.129:45355)
Aug 11 02:48:33 dist-test-slave-n326 krb5kdc[8539](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786416513, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:48:33.994602 9392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:35813
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:48:34.002458 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.236.148:47101/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:36883
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:48:34.144739 9802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:48:34.145059 9802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:48:34.145165 9802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:48:34.150667 9802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:48:34.150830 9802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:48:34.150962 9802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:48:34.156985 9802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.236.148:47101/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:36883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:48:34.158432 9802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:48:34.159608 9802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:48:34.170467 9812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:48:34.170467 9816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:48:34.171588 9814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:48:34.171670 9802 server_base.cc:1034] running on GCE node
I20260811 02:48:34.172176 9802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:48:34.172978 9802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:48:34.177317 9802 hybrid_clock.cc:648] HybridClock initialized: now 1786416514177229 us; error 96 us; skew 500 ppm
Aug 11 02:48:34 dist-test-slave-n326 krb5kdc[8539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416514, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:48:34.181494 9802 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:48:34.182652 9802 webserver.cc:460] Webserver started at http://127.21.236.130:43801/ using document root <none> and password file <none>
I20260811 02:48:34.183455 9802 fs_manager.cc:359] Metadata directory not provided
I20260811 02:48:34.183599 9802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:48:34.183877 9802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:48:34 dist-test-slave-n326 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416514, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:48:34.200991 9802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "aa977181b5cf46f6b97589dc4871a774"
format_stamp: "Formatted at 2026-08-11 02:48:34 on dist-test-slave-n326"
server_key: "17a03d98548becabdccad3dbff6db952"
server_key_iv: "f76eb8e28a27faf5d4355e1b8219a8d3"
server_key_version: "kuduclusterkey@0"
I20260811 02:48:34.201938 9802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "aa977181b5cf46f6b97589dc4871a774"
format_stamp: "Formatted at 2026-08-11 02:48:34 on dist-test-slave-n326"
server_key: "17a03d98548becabdccad3dbff6db952"
server_key_iv: "f76eb8e28a27faf5d4355e1b8219a8d3"
server_key_version: "kuduclusterkey@0"
I20260811 02:48:34.209314 9802 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260811 02:48:34.233114 9829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:48:34.235124 9802 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:48:34.235306 9802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "aa977181b5cf46f6b97589dc4871a774"
format_stamp: "Formatted at 2026-08-11 02:48:34 on dist-test-slave-n326"
server_key: "17a03d98548becabdccad3dbff6db952"
server_key_iv: "f76eb8e28a27faf5d4355e1b8219a8d3"
server_key_version: "kuduclusterkey@0"
I20260811 02:48:34.235476 9802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:48:34.252652 9802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:48:34.257936 9802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:48:34.258235 9802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:48:34.259164 9802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:48:34.260145 9802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:48:34.260233 9802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:48:34.260298 9802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:48:34.260329 9802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:48:34.274353 9802 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:42253
I20260811 02:48:34.275730 9802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:48:34.280205 10031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:42253 every 8 connection(s)
I20260811 02:48:34.284132 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 9802
I20260811 02:48:34.284282 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:48:34.284623 22450 external_mini_cluster.cc:1428] Setting key 3d8a17b27ea1c681f6e0f9f1d5479378
Aug 11 02:48:34 dist-test-slave-n326 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416514, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:48:34.293380 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.278082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:43611 (local address 127.21.236.190:36883)
0811 02:48:34.278226 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 02:48:34.278231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:48:34.280724 (+ 2493us) server_negotiation.cc:407] Connection header received
0811 02:48:34.282031 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:48:34.282035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:48:34.282108 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:48:34.282195 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:48:34.283507 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.284330 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.286014 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.286308 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.286445 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.289203 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:48:34.289207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:48:34.289209 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:48:34.289235 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:48:34.290881 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:48:34.291511 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:48:34.291515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:48:34.291518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:48:34.291607 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:48:34.292005 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:48:34.292009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:48:34.292011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:48:34.292207 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:48:34.292328 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 02:48:34.293056 (+ 728us) server_negotiation.cc:299] Negotiation successful
0811 02:48:34.293180 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:48:34.294808 10035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.278634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:36883 (local address 127.21.236.130:43611)
0811 02:48:34.280551 (+ 1917us) negotiation.cc:107] Waiting for socket to connect
0811 02:48:34.280600 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 02:48:34.281798 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:48:34.282377 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:48:34.282385 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:48:34.282701 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:48:34.283337 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:34.283355 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.284507 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.284511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:34.285860 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:34.285872 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.286483 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.286487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:34.286767 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.287498 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:48:34.287516 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:48:34.289005 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:48:34.291044 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:48:34.291050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:48:34.291093 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:48:34.291381 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:48:34.291717 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:48:34.291720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:48:34.291722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:48:34.291892 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:48:34.292326 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:48:34.292334 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:48:34.292723 (+ 389us) client_negotiation.cc:769] Sending connection context
0811 02:48:34.293647 (+ 924us) client_negotiation.cc:240] Negotiation successful
0811 02:48:34.293990 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1673,"thread_start_us":192,"threads_started":1}
I20260811 02:48:34.296105 10033 heartbeater.cc:347] Connected to a master server at 127.21.236.190:36883
I20260811 02:48:34.296366 10033 heartbeater.cc:464] Registering TS with master...
I20260811 02:48:34.296911 10033 heartbeater.cc:511] Master 127.21.236.190:36883 requested a full tablet report, sending...
I20260811 02:48:34.298067 9392 ts_manager.cc:194] Registered new tserver with Master: aa977181b5cf46f6b97589dc4871a774 (127.21.236.130:42253)
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
I20260811 02:48:34.298753 9392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:43611
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:48:34.328470 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.236.148:47101/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:36883
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:48:34 dist-test-slave-n326 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416513, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
W20260811 02:48:34.506129 10041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:48:34.506590 10041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:48:34.506716 10041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:48:34.512830 10041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:48:34.512984 10041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:48:34.513125 10041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:48:34.521081 10041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.236.148:47101/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:36883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:48:34.522886 10041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:48:34.524366 10041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:48:34.544070 10056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:48:34.546104 10050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:48:34.548584 10053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:48:34.549417 10041 server_base.cc:1034] running on GCE node
I20260811 02:48:34.550237 10041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:48:34.551173 10041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:48:34.558682 10041 hybrid_clock.cc:648] HybridClock initialized: now 1786416514558587 us; error 102 us; skew 500 ppm
Aug 11 02:48:34 dist-test-slave-n326 krb5kdc[8539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416514, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:48:34.562956 10041 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:48:34.564184 10041 webserver.cc:460] Webserver started at http://127.21.236.131:38711/ using document root <none> and password file <none>
I20260811 02:48:34.564963 10041 fs_manager.cc:359] Metadata directory not provided
I20260811 02:48:34.565117 10041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:48:34.565418 10041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:48:34 dist-test-slave-n326 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416514, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:48:34.589224 10041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "959ededd8bc84361ac144411c2b6831d"
format_stamp: "Formatted at 2026-08-11 02:48:34 on dist-test-slave-n326"
server_key: "476d96b9dd997ab8d85f7cf8d871d048"
server_key_iv: "1db738df7ec4b01680955939b1208cdf"
server_key_version: "kuduclusterkey@0"
I20260811 02:48:34.589874 10041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "959ededd8bc84361ac144411c2b6831d"
format_stamp: "Formatted at 2026-08-11 02:48:34 on dist-test-slave-n326"
server_key: "476d96b9dd997ab8d85f7cf8d871d048"
server_key_iv: "1db738df7ec4b01680955939b1208cdf"
server_key_version: "kuduclusterkey@0"
I20260811 02:48:34.594164 10041 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:48:34.614847 10067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:48:34.616441 10041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 02:48:34.616626 10041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "959ededd8bc84361ac144411c2b6831d"
format_stamp: "Formatted at 2026-08-11 02:48:34 on dist-test-slave-n326"
server_key: "476d96b9dd997ab8d85f7cf8d871d048"
server_key_iv: "1db738df7ec4b01680955939b1208cdf"
server_key_version: "kuduclusterkey@0"
I20260811 02:48:34.616777 10041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:48:34.629058 10041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:48:34.632333 10041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:48:34.632539 10041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:48:34.633214 10041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:48:34.634117 10041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:48:34.634166 10041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:48:34.634235 10041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:48:34.634267 10041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:48:34.646664 10041 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:45713
I20260811 02:48:34.646858 10254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:45713 every 8 connection(s)
I20260811 02:48:34.647950 10041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:48:34.649782 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 10041
I20260811 02:48:34.649932 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:48:34.650230 22450 external_mini_cluster.cc:1428] Setting key 6d47bc93f7b35092f27556d2f25bfa62
Aug 11 02:48:34 dist-test-slave-n326 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416514, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:48:34.667466 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.651732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:42601 (local address 127.21.236.190:36883)
0811 02:48:34.651856 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 02:48:34.651861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:48:34.652261 (+ 400us) server_negotiation.cc:407] Connection header received
0811 02:48:34.653449 (+ 1188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:48:34.653453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:48:34.653516 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:48:34.653612 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:48:34.655442 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.656388 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.658371 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.658765 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.658851 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.662554 (+ 3703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:48:34.662559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:48:34.662563 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:48:34.662597 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:48:34.664587 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:48:34.665374 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:48:34.665378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:48:34.665381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:48:34.665485 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:48:34.665944 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:48:34.665949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:48:34.665951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:48:34.666225 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:48:34.666342 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 02:48:34.667105 (+ 763us) server_negotiation.cc:299] Negotiation successful
0811 02:48:34.667254 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 02:48:34.668546 10260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.650594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:36883 (local address 127.21.236.131:42601)
0811 02:48:34.652061 (+ 1467us) negotiation.cc:107] Waiting for socket to connect
0811 02:48:34.652126 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 02:48:34.653192 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:48:34.653967 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:48:34.653979 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:48:34.654410 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:48:34.655238 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:34.655264 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.656578 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.656582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:34.658204 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:34.658218 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.659151 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.659157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:34.659386 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.660233 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:48:34.660258 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:48:34.662315 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:48:34.664808 (+ 2493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:48:34.664815 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:48:34.664833 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:48:34.665225 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:48:34.665606 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:48:34.665610 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:48:34.665612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:48:34.665813 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:48:34.666360 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:48:34.666367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:48:34.666718 (+ 351us) client_negotiation.cc:769] Sending connection context
0811 02:48:34.666965 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 02:48:34.667365 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1125,"thread_start_us":149,"threads_started":1}
I20260811 02:48:34.669865 10255 heartbeater.cc:347] Connected to a master server at 127.21.236.190:36883
I20260811 02:48:34.670126 10255 heartbeater.cc:464] Registering TS with master...
I20260811 02:48:34.670738 10255 heartbeater.cc:511] Master 127.21.236.190:36883 requested a full tablet report, sending...
I20260811 02:48:34.672003 9392 ts_manager.cc:194] Registered new tserver with Master: 959ededd8bc84361ac144411c2b6831d (127.21.236.131:45713)
I20260811 02:48:34.672714 9392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:42601
I20260811 02:48:34.676776 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:48:34.689285 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.679301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52302 (local address 127.21.236.190:36883)
0811 02:48:34.679449 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:48:34.679455 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:48:34.679698 (+ 243us) server_negotiation.cc:407] Connection header received
0811 02:48:34.679822 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:48:34.679827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:48:34.679878 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:48:34.679973 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:48:34.680856 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.681796 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.683024 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.683426 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.683572 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.685341 (+ 1769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:48:34.685345 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:48:34.685348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:48:34.685381 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:48:34.687368 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:48:34.687852 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:48:34.687856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:48:34.687858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:48:34.687958 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:48:34.688258 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:48:34.688262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:48:34.688264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:48:34.688506 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:48:34.688681 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 02:48:34.688934 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 02:48:34.689083 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 02:48:34.693221 9392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:48:34.698577 9392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:48:34.717883 10283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.709809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44790 (local address 127.21.236.130:42253)
0811 02:48:34.710366 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 02:48:34.710380 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:48:34.711945 (+ 1565us) server_negotiation.cc:407] Connection header received
0811 02:48:34.712229 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:48:34.712235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:48:34.712456 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:48:34.712593 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:48:34.713163 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.714370 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.716113 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.716960 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.717058 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.717189 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 02:48:34.717610 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 02:48:34.717718 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":151,"threads_started":1}
I20260811 02:48:34.718657 10285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.709656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:42253 (local address 127.0.0.1:44790)
0811 02:48:34.711794 (+ 2138us) negotiation.cc:107] Waiting for socket to connect
0811 02:48:34.711842 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 02:48:34.712057 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:48:34.712815 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:48:34.712821 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:48:34.712846 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:48:34.713018 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:34.713026 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.714523 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.714526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:34.715947 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:34.715959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.717284 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.717287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:34.717398 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.717435 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 02:48:34.718374 (+ 939us) client_negotiation.cc:240] Negotiation successful
0811 02:48:34.718513 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1965,"thread_start_us":112,"threads_started":1}
I20260811 02:48:34.720310 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.710963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:45355 (local address 127.0.0.1:52750)
0811 02:48:34.712906 (+ 1943us) negotiation.cc:107] Waiting for socket to connect
0811 02:48:34.712923 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:48:34.713049 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:48:34.713738 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:48:34.713742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:48:34.713761 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:48:34.713859 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:34.713867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.716390 (+ 2523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.716394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:34.717785 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:34.717796 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.719940 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.719944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:34.720031 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.720055 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:48:34.720122 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:48:34.720188 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":149,"threads_started":1}
I20260811 02:48:34.721143 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.711462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52750 (local address 127.21.236.129:45355)
0811 02:48:34.712811 (+ 1349us) server_negotiation.cc:206] Beginning negotiation
0811 02:48:34.712819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:48:34.713161 (+ 342us) server_negotiation.cc:407] Connection header received
0811 02:48:34.713247 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:48:34.713253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:48:34.713445 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:48:34.713587 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:48:34.714025 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.716235 (+ 2210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.717942 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.719696 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.719824 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.720855 (+ 1031us) server_negotiation.cc:1035] Waiting for connection context
0811 02:48:34.720910 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 02:48:34.721009 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1232,"thread_start_us":81,"threads_started":1}
I20260811 02:48:34.722548 9908 tablet_service.cc:1467] Processing CreateTablet for tablet 59b228f8b283451cb3caf0c97ee09a14 (DEFAULT_TABLE table=test-table [id=8c9c4279759f4f4daaefa83c219ee599]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:48:34.723959 9908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59b228f8b283451cb3caf0c97ee09a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:48:34.724512 9643 tablet_service.cc:1467] Processing CreateTablet for tablet 59b228f8b283451cb3caf0c97ee09a14 (DEFAULT_TABLE table=test-table [id=8c9c4279759f4f4daaefa83c219ee599]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:48:34.725071 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.711333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:45713 (local address 127.0.0.1:56832)
0811 02:48:34.719240 (+ 7907us) negotiation.cc:107] Waiting for socket to connect
0811 02:48:34.719255 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:48:34.719344 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:48:34.720273 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:48:34.720276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:48:34.720294 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:48:34.720379 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:34.720386 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.722122 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.722126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:34.723309 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:34.723323 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.724659 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.724663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:34.724753 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.724779 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 02:48:34.724849 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 02:48:34.724926 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7843,"thread_start_us":89,"threads_started":1}
I20260811 02:48:34.725616 9643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59b228f8b283451cb3caf0c97ee09a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:48:34.725623 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.711331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56832 (local address 127.21.236.131:45713)
0811 02:48:34.719630 (+ 8299us) server_negotiation.cc:206] Beginning negotiation
0811 02:48:34.719641 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:48:34.719665 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:48:34.719745 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:48:34.719750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:48:34.719967 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:48:34.720125 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:48:34.720583 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.721980 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.723499 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.724324 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.724462 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.724589 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 02:48:34.725361 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 02:48:34.725481 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8183,"thread_start_us":163,"threads_started":1}
I20260811 02:48:34.728475 10145 tablet_service.cc:1467] Processing CreateTablet for tablet 59b228f8b283451cb3caf0c97ee09a14 (DEFAULT_TABLE table=test-table [id=8c9c4279759f4f4daaefa83c219ee599]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:48:34.729532 10145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59b228f8b283451cb3caf0c97ee09a14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:48:34.737239 10295 tablet_bootstrap.cc:492] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3: Bootstrap starting.
I20260811 02:48:34.740136 10292 tablet_bootstrap.cc:492] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774: Bootstrap starting.
I20260811 02:48:34.741384 10295 tablet_bootstrap.cc:654] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3: Neither blocks nor log segments found. Creating new log.
I20260811 02:48:34.742398 10295 log.cc:824] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3: Log is configured to *not* fsync() on all Append() calls
I20260811 02:48:34.743295 10296 tablet_bootstrap.cc:492] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d: Bootstrap starting.
I20260811 02:48:34.743670 10292 tablet_bootstrap.cc:654] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774: Neither blocks nor log segments found. Creating new log.
I20260811 02:48:34.744648 10292 log.cc:824] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774: Log is configured to *not* fsync() on all Append() calls
I20260811 02:48:34.746308 10296 tablet_bootstrap.cc:654] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d: Neither blocks nor log segments found. Creating new log.
I20260811 02:48:34.747273 10296 log.cc:824] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d: Log is configured to *not* fsync() on all Append() calls
I20260811 02:48:34.749469 10295 tablet_bootstrap.cc:492] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3: No bootstrap required, opened a new log
I20260811 02:48:34.749760 10295 ts_tablet_manager.cc:1397] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.005s
I20260811 02:48:34.751272 10292 tablet_bootstrap.cc:492] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774: No bootstrap required, opened a new log
I20260811 02:48:34.751657 10292 ts_tablet_manager.cc:1397] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.004s
I20260811 02:48:34.754783 10295 raft_consensus.cc:348] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa977181b5cf46f6b97589dc4871a774" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 42253 } } peers { permanent_uuid: "bbb88e805e724ebd867029523e0937e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45355 } } peers { permanent_uuid: "959ededd8bc84361ac144411c2b6831d" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 45713 } }
I20260811 02:48:34.755378 10295 raft_consensus.cc:374] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:48:34.755496 10295 raft_consensus.cc:729] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbb88e805e724ebd867029523e0937e3, State: Initialized, Role: FOLLOWER
I20260811 02:48:34.756132 10295 consensus_queue.cc:260] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa977181b5cf46f6b97589dc4871a774" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 42253 } } peers { permanent_uuid: "bbb88e805e724ebd867029523e0937e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45355 } } peers { permanent_uuid: "959ededd8bc84361ac144411c2b6831d" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 45713 } }
I20260811 02:48:34.757613 10295 ts_tablet_manager.cc:1428] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260811 02:48:34.758095 9794 heartbeater.cc:503] Master 127.21.236.190:36883 was elected leader, sending a full tablet report...
I20260811 02:48:34.760075 10296 tablet_bootstrap.cc:492] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d: No bootstrap required, opened a new log
I20260811 02:48:34.760367 10296 ts_tablet_manager.cc:1397] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.000s
I20260811 02:48:34.764730 10296 raft_consensus.cc:348] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa977181b5cf46f6b97589dc4871a774" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 42253 } } peers { permanent_uuid: "bbb88e805e724ebd867029523e0937e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45355 } } peers { permanent_uuid: "959ededd8bc84361ac144411c2b6831d" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 45713 } }
I20260811 02:48:34.765205 10296 raft_consensus.cc:374] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:48:34.765318 10296 raft_consensus.cc:729] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959ededd8bc84361ac144411c2b6831d, State: Initialized, Role: FOLLOWER
I20260811 02:48:34.765889 10296 consensus_queue.cc:260] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa977181b5cf46f6b97589dc4871a774" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 42253 } } peers { permanent_uuid: "bbb88e805e724ebd867029523e0937e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45355 } } peers { permanent_uuid: "959ededd8bc84361ac144411c2b6831d" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 45713 } }
I20260811 02:48:34.767362 10296 ts_tablet_manager.cc:1428] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 02:48:34.767777 10255 heartbeater.cc:503] Master 127.21.236.190:36883 was elected leader, sending a full tablet report...
I20260811 02:48:34.775871 10292 raft_consensus.cc:348] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa977181b5cf46f6b97589dc4871a774" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 42253 } } peers { permanent_uuid: "bbb88e805e724ebd867029523e0937e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45355 } } peers { permanent_uuid: "959ededd8bc84361ac144411c2b6831d" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 45713 } }
I20260811 02:48:34.776289 10292 raft_consensus.cc:374] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:48:34.776396 10292 raft_consensus.cc:729] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa977181b5cf46f6b97589dc4871a774, State: Initialized, Role: FOLLOWER
I20260811 02:48:34.777002 10292 consensus_queue.cc:260] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa977181b5cf46f6b97589dc4871a774" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 42253 } } peers { permanent_uuid: "bbb88e805e724ebd867029523e0937e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45355 } } peers { permanent_uuid: "959ededd8bc84361ac144411c2b6831d" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 45713 } }
I20260811 02:48:34.778332 10292 ts_tablet_manager.cc:1428] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774: Time spent starting tablet: real 0.027s user 0.007s sys 0.000s
I20260811 02:48:34.778764 10033 heartbeater.cc:503] Master 127.21.236.190:36883 was elected leader, sending a full tablet report...
W20260811 02:48:34.779352 10034 tablet.cc:2369] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:48:34.784777 10301 raft_consensus.cc:482] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:48:34.785014 10301 raft_consensus.cc:504] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa977181b5cf46f6b97589dc4871a774" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 42253 } } peers { permanent_uuid: "bbb88e805e724ebd867029523e0937e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45355 } } peers { permanent_uuid: "959ededd8bc84361ac144411c2b6831d" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 45713 } }
I20260811 02:48:34.786302 10301 leader_election.cc:288] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa977181b5cf46f6b97589dc4871a774 (127.21.236.130:42253), 959ededd8bc84361ac144411c2b6831d (127.21.236.131:45713)
I20260811 02:48:34.792484 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.787624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:53707 (local address 127.21.236.131:45713)
0811 02:48:34.787784 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 02:48:34.787789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:48:34.787804 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:48:34.787860 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:48:34.787863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:48:34.787915 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:48:34.787992 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:48:34.788435 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.789220 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.791097 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.791700 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.791795 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.791850 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 02:48:34.792269 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 02:48:34.792353 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 02:48:34.792595 10283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.787483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:55009 (local address 127.21.236.130:42253)
0811 02:48:34.787768 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 02:48:34.787772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:48:34.787787 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:48:34.787833 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:48:34.787836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:48:34.787879 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:48:34.787958 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:48:34.789039 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.789738 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.791284 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.791897 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.791993 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.792040 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 02:48:34.792383 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 02:48:34.792474 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260811 02:48:34.792891 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.786822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:45713 (local address 127.21.236.129:53707)
0811 02:48:34.787448 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 02:48:34.787460 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:48:34.787552 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:48:34.788110 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:48:34.788113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:48:34.788131 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:48:34.788291 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:34.788299 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.789394 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.789397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:34.790919 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:34.790930 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.792005 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.792008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:34.792104 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.792136 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 02:48:34.792641 (+ 505us) client_negotiation.cc:240] Negotiation successful
0811 02:48:34.792720 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":113,"threads_started":1}
I20260811 02:48:34.793385 10313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.786575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:42253 (local address 127.21.236.129:55009)
0811 02:48:34.787103 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 02:48:34.787119 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:48:34.787232 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:48:34.788744 (+ 1512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:48:34.788747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:48:34.788767 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:48:34.788896 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:34.788903 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.789907 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.789910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:34.791134 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:48:34.791145 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.792137 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.792140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:48:34.792242 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.792264 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:48:34.793165 (+ 901us) client_negotiation.cc:240] Negotiation successful
0811 02:48:34.793242 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":146,"threads_started":1}
I20260811 02:48:34.793596 10156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59b228f8b283451cb3caf0c97ee09a14" candidate_uuid: "bbb88e805e724ebd867029523e0937e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959ededd8bc84361ac144411c2b6831d" is_pre_election: true
I20260811 02:48:34.793951 10156 raft_consensus.cc:2454] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbb88e805e724ebd867029523e0937e3 in term 0.
I20260811 02:48:34.794159 9943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59b228f8b283451cb3caf0c97ee09a14" candidate_uuid: "bbb88e805e724ebd867029523e0937e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa977181b5cf46f6b97589dc4871a774" is_pre_election: true
I20260811 02:48:34.794445 9943 raft_consensus.cc:2454] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbb88e805e724ebd867029523e0937e3 in term 0.
I20260811 02:48:34.794502 9572 leader_election.cc:302] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 959ededd8bc84361ac144411c2b6831d, bbb88e805e724ebd867029523e0937e3; no voters:
I20260811 02:48:34.794813 10301 raft_consensus.cc:2781] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:48:34.794901 10301 raft_consensus.cc:482] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:48:34.794947 10301 raft_consensus.cc:3037] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:48:34.796623 10301 raft_consensus.cc:504] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa977181b5cf46f6b97589dc4871a774" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 42253 } } peers { permanent_uuid: "bbb88e805e724ebd867029523e0937e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45355 } } peers { permanent_uuid: "959ededd8bc84361ac144411c2b6831d" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 45713 } }
I20260811 02:48:34.797025 10301 leader_election.cc:288] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [CANDIDATE]: Term 1 election: Requested vote from peers aa977181b5cf46f6b97589dc4871a774 (127.21.236.130:42253), 959ededd8bc84361ac144411c2b6831d (127.21.236.131:45713)
I20260811 02:48:34.797363 9943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59b228f8b283451cb3caf0c97ee09a14" candidate_uuid: "bbb88e805e724ebd867029523e0937e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa977181b5cf46f6b97589dc4871a774"
I20260811 02:48:34.797434 10156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59b228f8b283451cb3caf0c97ee09a14" candidate_uuid: "bbb88e805e724ebd867029523e0937e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "959ededd8bc84361ac144411c2b6831d"
I20260811 02:48:34.797504 9943 raft_consensus.cc:3037] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:48:34.797566 10156 raft_consensus.cc:3037] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:48:34.798746 9943 raft_consensus.cc:2454] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbb88e805e724ebd867029523e0937e3 in term 1.
I20260811 02:48:34.798867 10156 raft_consensus.cc:2454] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbb88e805e724ebd867029523e0937e3 in term 1.
I20260811 02:48:34.799162 9573 leader_election.cc:302] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa977181b5cf46f6b97589dc4871a774, bbb88e805e724ebd867029523e0937e3; no voters:
I20260811 02:48:34.799463 10301 raft_consensus.cc:2781] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:48:34.799687 10301 raft_consensus.cc:686] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [term 1 LEADER]: Becoming Leader. State: Replica: bbb88e805e724ebd867029523e0937e3, State: Running, Role: LEADER
I20260811 02:48:34.800098 10301 consensus_queue.cc:237] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa977181b5cf46f6b97589dc4871a774" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 42253 } } peers { permanent_uuid: "bbb88e805e724ebd867029523e0937e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45355 } } peers { permanent_uuid: "959ededd8bc84361ac144411c2b6831d" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 45713 } }
I20260811 02:48:34.803813 9385 catalog_manager.cc:5495] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bbb88e805e724ebd867029523e0937e3 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "bbb88e805e724ebd867029523e0937e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa977181b5cf46f6b97589dc4871a774" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 42253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbb88e805e724ebd867029523e0937e3" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 45355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "959ededd8bc84361ac144411c2b6831d" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 45713 } health_report { overall_health: UNKNOWN } } }
I20260811 02:48:34.834707 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.829618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52760 (local address 127.21.236.129:45355)
0811 02:48:34.829742 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 02:48:34.829747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:48:34.829766 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:48:34.829971 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:48:34.829975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:48:34.830049 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:48:34.830165 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:48:34.830582 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.831428 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.832760 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:48:34.833105 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:48:34.833204 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:48:34.833663 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:48:34.834268 (+ 605us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:48:34.834417 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 02:48:34.834491 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 02:48:34.834560 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 02:48:34.843222 10156 raft_consensus.cc:1261] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d [term 1 FOLLOWER]: Refusing update from remote peer bbb88e805e724ebd867029523e0937e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:48:34.844064 10318 consensus_queue.cc:1035] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "959ededd8bc84361ac144411c2b6831d" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 45713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:48:34.844669 9943 raft_consensus.cc:1261] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774 [term 1 FOLLOWER]: Refusing update from remote peer bbb88e805e724ebd867029523e0937e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:48:34.845414 10318 consensus_queue.cc:1035] T 59b228f8b283451cb3caf0c97ee09a14 P bbb88e805e724ebd867029523e0937e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa977181b5cf46f6b97589dc4871a774" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 42253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 11 02:48:34 dist-test-slave-n326 krb5kdc[8539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416513, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:48:34.859864 10326 mvcc.cc:205] Tried to move back new op lower bound from 7317162044789288960 to 7317162044624486400. Current Snapshot: MvccSnapshot[applied={T|T < 7317162044789288960}]
I20260811 02:48:34.870975 9392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:48:34.871335 9392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:48:34.874150 9392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91b08c5f96f84777bb81f65a2f7c9e28: Sending DeleteTablet for 3 replicas of tablet 59b228f8b283451cb3caf0c97ee09a14
I20260811 02:48:34.875005 9908 tablet_service.cc:1514] Processing DeleteTablet for tablet 59b228f8b283451cb3caf0c97ee09a14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:48:34 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:44790
I20260811 02:48:34.875387 9643 tablet_service.cc:1514] Processing DeleteTablet for tablet 59b228f8b283451cb3caf0c97ee09a14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:48:34 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:52750
I20260811 02:48:34.877146 10145 tablet_service.cc:1514] Processing DeleteTablet for tablet 59b228f8b283451cb3caf0c97ee09a14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:48:34 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:56832
I20260811 02:48:34.878827 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 9530
I20260811 02:48:34.900050 10342 tablet_replica.cc:330] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d: stopping tablet replica
I20260811 02:48:34.901888 10342 raft_consensus.cc:2229] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:48:34.902937 10342 raft_consensus.cc:2258] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:48:34.904038 10338 tablet_replica.cc:330] T 59b228f8b283451cb3caf0c97ee09a14 P aa977181b5cf46f6b97589dc4871a774: stopping tablet replica
I20260811 02:48:34.906883 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 9802
W20260811 02:48:34.906945 9364 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:45355 (error 108)
I20260811 02:48:34.915828 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.915390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:45355 (local address 127.0.0.1:52766)
0811 02:48:34.915633 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 02:48:34.915715 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:45355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
W20260811 02:48:34.916631 9364 catalog_manager.cc:4584] TS bbb88e805e724ebd867029523e0937e3 (127.21.236.129:45355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 59b228f8b283451cb3caf0c97ee09a14: Network error: Client connection negotiation failed: client connection to 127.21.236.129:45355: connect: Connection refused (error 111)
I20260811 02:48:34.917480 10342 ts_tablet_manager.cc:1905] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:48:34.921206 10342 ts_tablet_manager.cc:1918] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:48:34.921444 10342 log.cc:1196] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/59b228f8b283451cb3caf0c97ee09a14
I20260811 02:48:34.921994 10342 ts_tablet_manager.cc:1939] T 59b228f8b283451cb3caf0c97ee09a14 P 959ededd8bc84361ac144411c2b6831d: Deleting consensus metadata
I20260811 02:48:34.922549 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 10041
I20260811 02:48:34.926131 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:48:34.925792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:42253 (local address 127.0.0.1:44794)
0811 02:48:34.925991 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 02:48:34.926044 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.130:42253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260811 02:48:34.937942 10282 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:48:34.937191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:45713 (local address 127.0.0.1:56836)
0811 02:48:34.937388 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 02:48:34.937405 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:48:34.937515 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:48:34.937781 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.131:45713: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 02:48:34.938634 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 9327
I20260811 02:48:34.953078 22450 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 02:48:35.303691 22450 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 02:48:35.303826 22450 mini_ranger.cc:66] Stopping Ranger...
I20260811 02:48:35.660207 22450 mini_ranger.cc:68] Stopped Ranger
2026-08-11 02:48:35.660 UTC [8593] LOG: received smart shutdown request
2026-08-11 02:48:35.664 UTC [8593] LOG: background worker "logical replication launcher" (PID 8614) exited with exit code 1
2026-08-11 02:48:35.665 UTC [8603] LOG: shutting down
2026-08-11 02:48:35.855 UTC [8593] LOG: database system is shut down
2026-08-11T02:48:35Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50075 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:48:36 dist-test-slave-n326 krb5kdc[10359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:48:36 dist-test-slave-n326 krb5kdc[10359](info): set up 2 sockets
Aug 11 02:48:36 dist-test-slave-n326 krb5kdc[10359](info): commencing operation
krb5kdc: starting...
W20260811 02:48:37.295552 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.004s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:48:37 dist-test-slave-n326 krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:48:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:48:37Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/rangeradmin_127.21.236.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/rangeradmin_127.21.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/rangerlookup_127.21.236.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/rangerlookup_127.21.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/HTTP_127.21.236.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/HTTP_127.21.236.148@KRBTEST.COM.keytab.
I20260811 02:48:37.474464 22450 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskQxaM5w/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/postgres -l logfile start
2026-08-11 02:48:40.090 UTC [10407] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 02:48:40.090 UTC [10407] LOG: listening on IPv4 address "127.21.236.148", port 37567
2026-08-11 02:48:40.094 UTC [10407] LOG: listening on Unix socket "/tmp/.s.PGSQL.37567"
2026-08-11 02:48:40.116 UTC [10413] LOG: database system was shut down at 2026-08-11 02:48:38 UTC
2026-08-11 02:48:40.121 UTC [10407] LOG: database system is ready to accept connections
I20260811 02:48:41.390829 22450 mini_postgres.cc:96] Postgres bound to 37567
2026-08-11 02:48:41.399 UTC [10424] FATAL: database "slave" does not exist
127.21.236.148:37567 - accepting connections
I20260811 02:48:41.400471 22450 mini_ranger.cc:161] Starting Ranger...
I20260811 02:48:41.417824 22450 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 02:48:41.934824 22450 mini_ranger.cc:87] Created ranger Postgres database
I20260811 02:48:41.934926 22450 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-admin
2026-08-11 02:48:42,236 [I] DB FLAVOR :POSTGRES
2026-08-11 02:48:42,237 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 02:48:42,237 [I] Checking connection..
2026-08-11 02:48:42,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 02:48:42,518 [I] Checking connection passed.
2026-08-11 02:48:42,519 [I] --------- Verifying version history table ---------
2026-08-11 02:48:42,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:48:42,790 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 02:48:42,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:48:43,054 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 02:48:43,054 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 02:48:43,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 02:48:43.302 UTC [10534] WARNING: there is no transaction in progress
2026-08-11 02:48:43,312 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 02:48:43,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 02:48:43,595 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 02:48:43,595 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 02:48:43,596 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 02:48:43,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 02:48:44,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-n326.c.gcp-upstream.internal','N') ;"
2026-08-11 02:48:44,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 02:48:44,667 [I] Table x_portal_user does not exist in database ranger
2026-08-11 02:48:44,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 02:48:44,934 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 02:48:44,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 02:48:45,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 02:48:45,512 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 02:48:45,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 02:48:46.560 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.576 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.593 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.608 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.624 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.644 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.711 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.718 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.734 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.745 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.755 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.770 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.780 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.788 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.795 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:46.802 UTC [10719] WARNING: there is no transaction in progress
2026-08-11 02:48:47,483 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 02:48:47,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 02:48:47,708 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 02:48:47,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 02:48:47,955 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 02:48:47,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 02:48:48,221 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 02:48:48,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 02:48:48,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 02:48:48,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-n326.c.gcp-upstream.internal';"
2026-08-11 02:48:48,999 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 02:48:48,999 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 02:48:49,251 [I] DB_PATCHES have already been applied
I20260811 02:48:49.257917 22450 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-admin:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskQxaM5w/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 02:48:49.258023 22450 mini_ranger.cc:193] Using host: 127.21.236.148
I20260811 02:48:49.262712 22450 mini_ranger.cc:227] Ranger admin URL: http://127.21.236.148:51451
Aug 11, 2026 2:48:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:48:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:48:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 2:48:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 2:48:49 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 2:48:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 2:48:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 02:48:50.809499 22450 mini_ranger.cc:160] Time spent starting Ranger: real 9.409s user 0.000s sys 0.013s
Aug 11, 2026 2:49:11 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 2:49:11 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 2:49:11 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 2:49:11 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 2:49:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 2:49:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 2:49:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 2:49:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 2:49:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 2:49:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 2:49:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 2:49:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 2:49:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 2:49:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 2:49:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 2:49:11 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 2:49:11 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 2:49:12 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 02:49:13.259267 22450 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.236.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.236.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/rangerkms_127.21.236.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.236.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/rangerkms_127.21.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.236.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.236.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.236.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/HTTP_127.21.236.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.236.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/HTTP_127.21.236.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 02:49:13 dist-test-slave-n326 krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416553, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 02:49:13.353900 22450 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 02:49:13.365808 22450 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 02:49:14.257433 22450 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 02:49:14.257563 22450 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-kms
I20260811 02:49:14.257591 22450 mini_ranger_kms.cc:227] Using postgres at 127.21.236.148:37567
2026-08-11 02:49:14,430 [I] DB FLAVOR :POSTGRES
2026-08-11 02:49:14,430 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 02:49:14,431 [I] Checking connection
2026-08-11 02:49:14,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 02:49:14,674 [I] connection success
2026-08-11 02:49:14,674 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 02:49:14,675 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 02:49:14,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 02:49:14,929 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 02:49:14,929 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 02:49:14,930 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 02:49:14,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.236.148:37567/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 02:49:15,199 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 02:49:15.725291 22450 mini_ranger_kms.cc:310] Created kms service
I20260811 02:49:15.956708 22450 mini_ranger_kms.cc:326] Created kudu user
I20260811 02:49:16.003556 22450 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 02:49:16 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416553, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:49:16.454440 22450 mini_ranger_kms.cc:384] Added ranger policy
I20260811 02:49:16.454715 22450 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-kms:/tmp/dist-test-taskQxaM5w/build/debug/bin/postgresql.jar:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskQxaM5w/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 02:49:16.454746 22450 mini_ranger_kms.cc:240] Using host: 127.21.236.148
I20260811 02:49:16.459801 22450 mini_ranger_kms.cc:276] Ranger KMS PID: 11016
I20260811 02:49:16.459888 22450 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.236.148:51983
Aug 11, 2026 2:49:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:49:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:49:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 2:49:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 2:49:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 2:49:18 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 02:49:18 dist-test-slave-n326 krb5kdc[10359](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786416558, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.236.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 02:49:19 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416558, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.236.148@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:49:19.481411 22450 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.128s user 0.014s sys 0.145s
I20260811 02:49:19.481531 22450 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 02:49:19.481592 22450 mini_ranger_kms.cc:401] 127.21.236.148:51983/kms/v1/keys
Aug 11 02:49:19 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416558, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.236.148@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
Aug 11 02:49:21 dist-test-slave-n326 krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:49:22.042078 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:44197
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:44197
--ranger_config_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.236.148:51983/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:22.160388 11112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:22.160688 11112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:22.160730 11112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:22.165040 11112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:49:22.165122 11112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:22.165143 11112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:22.165160 11112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:49:22.165176 11112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:49:22.171038 11112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.236.148:51983/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:44197
--ranger_config_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:44197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:22.172219 11112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:22.173280 11112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:22.182592 11120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:22.182588 11122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:22.182765 11112 server_base.cc:1034] running on GCE node
W20260811 02:49:22.182585 11124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:22.183524 11112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:22.184696 11112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:22.185884 11112 hybrid_clock.cc:648] HybridClock initialized: now 1786416562185852 us; error 59 us; skew 500 ppm
Aug 11 02:49:22 dist-test-slave-n326 krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416562, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:22.193748 11112 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:49:22.194880 11112 webserver.cc:460] Webserver started at http://127.21.236.190:37117/ using document root <none> and password file <none>
I20260811 02:49:22.195525 11112 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:22.195588 11112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:22.195824 11112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:49:22 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416562, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:49:22.257920 11112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "86e9b70cb78a4eac86634f14b9a7f619"
format_stamp: "Formatted at 2026-08-11 02:49:22 on dist-test-slave-n326"
server_key: "431fc6d187d3664fbea2dc72956f2d39"
server_key_iv: "1ea66dfe61bc71e5bfab5b8817a4737a"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:22.258692 11112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "86e9b70cb78a4eac86634f14b9a7f619"
format_stamp: "Formatted at 2026-08-11 02:49:22 on dist-test-slave-n326"
server_key: "431fc6d187d3664fbea2dc72956f2d39"
server_key_iv: "1ea66dfe61bc71e5bfab5b8817a4737a"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:22.262981 11112 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:49:22.279425 11141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:22.280925 11112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:49:22.281095 11112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "86e9b70cb78a4eac86634f14b9a7f619"
format_stamp: "Formatted at 2026-08-11 02:49:22 on dist-test-slave-n326"
server_key: "431fc6d187d3664fbea2dc72956f2d39"
server_key_iv: "1ea66dfe61bc71e5bfab5b8817a4737a"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:22.281250 11112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:22.306581 11112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:22.309872 11112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:22.310096 11112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:22.318661 11112 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:44197
I20260811 02:49:22.318662 11226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:44197 every 8 connection(s)
I20260811 02:49:22.319830 11112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:49:22.321655 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 11112
I20260811 02:49:22.321801 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:49:22.322161 22450 external_mini_cluster.cc:1428] Setting key 6935ecfbadf94c659488f658bf450713
I20260811 02:49:22.322976 11228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:22.330168 11228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619: Bootstrap starting.
Aug 11 02:49:22 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:22.332818 11228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:22.333817 11228 log.cc:824] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:22.336139 11228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619: No bootstrap required, opened a new log
I20260811 02:49:22.338416 11231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:22.323723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34296 (local address 127.21.236.190:44197)
0811 02:49:22.324212 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:22.324232 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:22.324276 (+ 44us) server_negotiation.cc:407] Connection header received
0811 02:49:22.325204 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:22.325229 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:22.325669 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:22.326045 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:22.326795 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:22.328294 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:22.329293 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:22.329976 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:22.330140 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:22.333007 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:22.333015 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:22.333037 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:22.333074 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:22.335541 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:22.336040 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:22.336045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:22.336049 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:22.336156 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:22.336471 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:22.336475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:22.336476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:22.336892 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:22.337111 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:22.337418 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 02:49:22.337720 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":161,"threads_started":1}
I20260811 02:49:22.339336 11228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e9b70cb78a4eac86634f14b9a7f619" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 44197 } }
I20260811 02:49:22.339582 11228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:22.339664 11228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86e9b70cb78a4eac86634f14b9a7f619, State: Initialized, Role: FOLLOWER
I20260811 02:49:22.340095 11228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e9b70cb78a4eac86634f14b9a7f619" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 44197 } }
I20260811 02:49:22.340250 11228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:49:22.340337 11228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:49:22.340456 11228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:22.341507 11228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e9b70cb78a4eac86634f14b9a7f619" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 44197 } }
I20260811 02:49:22.341849 11228 leader_election.cc:302] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86e9b70cb78a4eac86634f14b9a7f619; no voters:
I20260811 02:49:22.342182 11228 leader_election.cc:288] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:49:22.342286 11236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:49:22.342550 11236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 1 LEADER]: Becoming Leader. State: Replica: 86e9b70cb78a4eac86634f14b9a7f619, State: Running, Role: LEADER
I20260811 02:49:22.342944 11236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e9b70cb78a4eac86634f14b9a7f619" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 44197 } }
I20260811 02:49:22.343454 11228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:49:22.344760 11240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86e9b70cb78a4eac86634f14b9a7f619. Latest consensus state: current_term: 1 leader_uuid: "86e9b70cb78a4eac86634f14b9a7f619" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e9b70cb78a4eac86634f14b9a7f619" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 44197 } } }
I20260811 02:49:22.344892 11240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:22.345165 11238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86e9b70cb78a4eac86634f14b9a7f619" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e9b70cb78a4eac86634f14b9a7f619" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 44197 } } }
I20260811 02:49:22.345263 11238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:22.345280 11251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:49:22.348476 11251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:49:22.348928 11228 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 02:49:22.354526 11251 catalog_manager.cc:1347] Generated new cluster ID: d8695b63c401464a901831321dbbd0df
I20260811 02:49:22.354610 11251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:49:22.372745 11251 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:49:22.374219 11251 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:49:22.380594 11251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619: Generated new TSK 0
I20260811 02:49:22.381428 11251 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:49:23.075749 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.236.148:51983/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:44197
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:23.205626 11306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:23.205942 11306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:23.206022 11306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:23.210397 11306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:23.210486 11306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:23.210598 11306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:49:23.216629 11306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.236.148:51983/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:44197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:23.217901 11306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:23.219038 11306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:23.229872 11316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:23.229890 11313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:23.229890 11312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:23.231487 11306 server_base.cc:1034] running on GCE node
I20260811 02:49:23.231945 11306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:23.232648 11306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:23.233836 11306 hybrid_clock.cc:648] HybridClock initialized: now 1786416563233795 us; error 64 us; skew 500 ppm
Aug 11 02:49:23 dist-test-slave-n326 krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:23.237111 11306 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:49:23.238032 11306 webserver.cc:460] Webserver started at http://127.21.236.129:39039/ using document root <none> and password file <none>
I20260811 02:49:23.238641 11306 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:23.238739 11306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:23.238981 11306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:49:23 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:49:23.257790 11306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "d595b681c94a4c3cb88a3de3740f3881"
format_stamp: "Formatted at 2026-08-11 02:49:23 on dist-test-slave-n326"
server_key: "d93a0ff2115b16821eeaa300f8f52e58"
server_key_iv: "49e6365175fa06566a00d2b2c3e2c6a1"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:23.258565 11306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "d595b681c94a4c3cb88a3de3740f3881"
format_stamp: "Formatted at 2026-08-11 02:49:23 on dist-test-slave-n326"
server_key: "d93a0ff2115b16821eeaa300f8f52e58"
server_key_iv: "49e6365175fa06566a00d2b2c3e2c6a1"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:23.266846 11306 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260811 02:49:23.286880 11334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:23.288537 11306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:49:23.288707 11306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "d595b681c94a4c3cb88a3de3740f3881"
format_stamp: "Formatted at 2026-08-11 02:49:23 on dist-test-slave-n326"
server_key: "d93a0ff2115b16821eeaa300f8f52e58"
server_key_iv: "49e6365175fa06566a00d2b2c3e2c6a1"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:23.288867 11306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:23.309753 11306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:23.313112 11306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:23.313334 11306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:23.314183 11306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:23.315254 11306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:23.315326 11306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:23.315421 11306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:23.315456 11306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:23.334836 11306 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:41127
I20260811 02:49:23.334877 11514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:41127 every 8 connection(s)
I20260811 02:49:23.336225 11306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:49:23.345862 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 11306
I20260811 02:49:23.346047 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:49:23.346401 22450 external_mini_cluster.cc:1428] Setting key f31025d83b713ca834c0892ad2df0472
Aug 11 02:49:23 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:23.358171 11522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:23.338733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:44197 (local address 127.21.236.129:55885)
0811 02:49:23.339246 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:23.339300 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:23.340203 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:23.341004 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:23.341014 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:23.341434 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:23.342222 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:23.342247 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:23.343578 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:23.343582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:23.345037 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:23.345050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:23.345896 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:23.345901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:23.346280 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:23.347131 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:23.347152 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:23.348998 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:23.351448 (+ 2450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:23.351455 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:23.351473 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:23.351857 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:23.352220 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:23.352224 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:23.352226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:23.352435 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:23.355891 (+ 3456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:23.355900 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:23.356297 (+ 397us) client_negotiation.cc:769] Sending connection context
0811 02:49:23.356555 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 02:49:23.356964 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":145,"threads_started":1}
I20260811 02:49:23.359009 11519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:23.338305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:55885 (local address 127.21.236.190:44197)
0811 02:49:23.338678 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:23.338689 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:23.339418 (+ 729us) server_negotiation.cc:407] Connection header received
0811 02:49:23.340450 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:23.340455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:23.340520 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:23.340644 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:23.342424 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:23.343374 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:23.345203 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:23.345583 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:23.345700 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:23.349215 (+ 3515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:23.349219 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:23.349222 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:23.349248 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:23.351306 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:23.351986 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:23.351990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:23.351991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:23.352090 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:23.352559 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:23.352563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:23.352565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:23.352806 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:23.352910 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:23.358660 (+ 5750us) server_negotiation.cc:299] Negotiation successful
0811 02:49:23.358821 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":135,"threads_started":1}
I20260811 02:49:23.360982 11516 heartbeater.cc:347] Connected to a master server at 127.21.236.190:44197
I20260811 02:49:23.361255 11516 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:23.361871 11516 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
I20260811 02:49:23.363811 11165 ts_manager.cc:194] Registered new tserver with Master: d595b681c94a4c3cb88a3de3740f3881 (127.21.236.129:41127)
I20260811 02:49:23.365428 11165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:55885
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:49:23.399537 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.236.148:51983/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:44197
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:49:23 dist-test-slave-n326 krb5kdc[10359](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786416563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 02:49:23.575941 11534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:23.576367 11534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:23.576458 11534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:23.583328 11534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:23.583428 11534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:23.583528 11534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:49:23.590464 11534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.236.148:51983/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:44197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:23.591799 11534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:23.593138 11534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:23.611933 11545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:23.612452 11544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:23.612782 11548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:23.612812 11534 server_base.cc:1034] running on GCE node
I20260811 02:49:23.613415 11534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:23.614252 11534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:23.617568 11534 hybrid_clock.cc:648] HybridClock initialized: now 1786416563617470 us; error 121 us; skew 500 ppm
Aug 11 02:49:23 dist-test-slave-n326 krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:23.621824 11534 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:49:23.623118 11534 webserver.cc:460] Webserver started at http://127.21.236.130:37607/ using document root <none> and password file <none>
I20260811 02:49:23.623939 11534 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:23.624042 11534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:23.624301 11534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:49:23 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:49:23.643126 11534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "34986516f5844695954d9ecab88fa881"
format_stamp: "Formatted at 2026-08-11 02:49:23 on dist-test-slave-n326"
server_key: "6fa3a7154391115d24983dc5a81fc670"
server_key_iv: "a2789d79baed4f5a1b681e2eaeb9071b"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:23.644349 11534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "34986516f5844695954d9ecab88fa881"
format_stamp: "Formatted at 2026-08-11 02:49:23 on dist-test-slave-n326"
server_key: "6fa3a7154391115d24983dc5a81fc670"
server_key_iv: "a2789d79baed4f5a1b681e2eaeb9071b"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:23.660998 11534 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.004s sys 0.000s
I20260811 02:49:23.687638 11556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:23.689601 11534 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.000s
I20260811 02:49:23.689771 11534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "34986516f5844695954d9ecab88fa881"
format_stamp: "Formatted at 2026-08-11 02:49:23 on dist-test-slave-n326"
server_key: "6fa3a7154391115d24983dc5a81fc670"
server_key_iv: "a2789d79baed4f5a1b681e2eaeb9071b"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:23.690388 11534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:23.724843 11534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:23.729084 11534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:23.729302 11534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:23.730008 11534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:23.731189 11534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:23.731259 11534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:23.731325 11534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:23.731357 11534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:23.744833 11749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:41343 every 8 connection(s)
I20260811 02:49:23.744872 11534 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:41343
I20260811 02:49:23.746399 11534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:49:23.747890 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 11534
I20260811 02:49:23.748041 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:49:23.748406 22450 external_mini_cluster.cc:1428] Setting key 45898d3f69bb3b770eb217ef8235ec5a
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
Aug 11 02:49:23 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:23.775300 11519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:23.758576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:35945 (local address 127.21.236.190:44197)
0811 02:49:23.758754 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:23.758760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:23.759708 (+ 948us) server_negotiation.cc:407] Connection header received
0811 02:49:23.761478 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:23.761495 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:23.761574 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:23.761641 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:23.763762 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:23.764753 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:23.766686 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:23.766951 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:23.767124 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:23.770704 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:23.770709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:23.770712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:23.770746 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:23.772741 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:23.773400 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:23.773403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:23.773405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:23.773519 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:23.773923 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:23.773926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:23.773929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:23.774192 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:23.774307 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:23.774932 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 02:49:23.775123 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:49:23.776168 11753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:23.757245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:44197 (local address 127.21.236.130:35945)
0811 02:49:23.759506 (+ 2261us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:23.759580 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:23.761129 (+ 1549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:23.762206 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:23.762220 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:23.762674 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:23.763534 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:23.763563 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:23.764954 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:23.764961 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:23.766501 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:23.766514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:23.767197 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:23.767203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:23.767438 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:23.768348 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:23.768371 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:23.770364 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:23.772900 (+ 2536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:23.772908 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:23.772930 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:23.773263 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:23.773626 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:23.773628 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:23.773630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:23.773806 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:23.774324 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:23.774331 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:23.774631 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 02:49:23.774825 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 02:49:23.775284 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1970,"thread_start_us":208,"threads_started":1}
I20260811 02:49:23.777657 11750 heartbeater.cc:347] Connected to a master server at 127.21.236.190:44197
I20260811 02:49:23.777961 11750 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:23.778637 11750 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
I20260811 02:49:23.780117 11165 ts_manager.cc:194] Registered new tserver with Master: 34986516f5844695954d9ecab88fa881 (127.21.236.130:41343)
I20260811 02:49:23.781034 11165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:35945
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:49:23.791731 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.236.148:51983/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:44197
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:23.967106 11757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:23.967576 11757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:23.967694 11757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Aug 11 02:49:23 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416563, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
W20260811 02:49:23.973927 11757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:23.974011 11757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:23.974097 11757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:49:23.990307 11757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.236.148:51983/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:44197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:23.991782 11757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:23.993106 11757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:24.006963 11768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:24.007268 11769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:24.007730 11772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:24.008348 11757 server_base.cc:1034] running on GCE node
I20260811 02:49:24.009092 11757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:24.010035 11757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:24.017433 11757 hybrid_clock.cc:648] HybridClock initialized: now 1786416564017192 us; error 266 us; skew 500 ppm
Aug 11 02:49:24 dist-test-slave-n326 krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416564, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:24.021574 11757 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:49:24.022845 11757 webserver.cc:460] Webserver started at http://127.21.236.131:36087/ using document root <none> and password file <none>
I20260811 02:49:24.023944 11757 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:24.024057 11757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:24.024392 11757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 02:49:24 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416564, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:49:24.056103 11757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "263218133f274ab8a56a9098e559edaa"
format_stamp: "Formatted at 2026-08-11 02:49:24 on dist-test-slave-n326"
server_key: "00de4d7456b815990eaf0b8ee591cf4c"
server_key_iv: "76530a2101ae7e341407a5c5105bdf19"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:24.056857 11757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "263218133f274ab8a56a9098e559edaa"
format_stamp: "Formatted at 2026-08-11 02:49:24 on dist-test-slave-n326"
server_key: "00de4d7456b815990eaf0b8ee591cf4c"
server_key_iv: "76530a2101ae7e341407a5c5105bdf19"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:24.064199 11757 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260811 02:49:24.090165 11783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:24.091938 11757 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 02:49:24.092118 11757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "263218133f274ab8a56a9098e559edaa"
format_stamp: "Formatted at 2026-08-11 02:49:24 on dist-test-slave-n326"
server_key: "00de4d7456b815990eaf0b8ee591cf4c"
server_key_iv: "76530a2101ae7e341407a5c5105bdf19"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:24.092275 11757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:24.113615 11757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:24.116930 11757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:24.117236 11757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:24.117951 11757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:24.118932 11757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:24.119004 11757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:24.119145 11757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:24.119195 11757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:24.131215 11757 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:35223
I20260811 02:49:24.132509 11757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:49:24.134383 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 11757
I20260811 02:49:24.134549 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:49:24.134871 22450 external_mini_cluster.cc:1428] Setting key 2af4675e7c923fb3248521a4cfbbe566
I20260811 02:49:24.137580 12000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:35223 every 8 connection(s)
Aug 11 02:49:24 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416564, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:24.152874 11519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:24.135339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:48267 (local address 127.21.236.190:44197)
0811 02:49:24.135495 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:24.135499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:24.137659 (+ 2160us) server_negotiation.cc:407] Connection header received
0811 02:49:24.139014 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:24.139018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:24.139117 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:24.139208 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:24.140908 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:24.141787 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:24.143746 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:24.144121 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:24.144245 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:24.147501 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:24.147505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:24.147508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:24.147539 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:24.149975 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:24.150755 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:24.150759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:24.150761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:24.150863 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:24.151392 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:24.151396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:24.151398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:24.151668 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:24.151775 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:24.152537 (+ 762us) server_negotiation.cc:299] Negotiation successful
0811 02:49:24.152693 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 02:49:24.154094 12005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:24.136313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:44197 (local address 127.21.236.131:48267)
0811 02:49:24.137489 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:24.137558 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:24.138749 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:24.139559 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:24.139569 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:24.139959 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:24.140717 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:24.140737 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:24.141943 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:24.141947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:24.143555 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:24.143568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:24.144309 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:24.144314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:24.144553 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:24.145315 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:24.145338 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:24.147267 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:24.150127 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:24.150135 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:24.150158 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:24.150623 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:24.150989 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:24.150993 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:24.150996 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:24.151277 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:24.151807 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:24.151816 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:24.152205 (+ 389us) client_negotiation.cc:769] Sending connection context
0811 02:49:24.152453 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 02:49:24.152869 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"thread_start_us":157,"threads_started":1}
I20260811 02:49:24.155534 12001 heartbeater.cc:347] Connected to a master server at 127.21.236.190:44197
I20260811 02:49:24.155843 12001 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:24.156595 12001 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
I20260811 02:49:24.157994 11165 ts_manager.cc:194] Registered new tserver with Master: 263218133f274ab8a56a9098e559edaa (127.21.236.131:35223)
I20260811 02:49:24.158831 11165 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:48267
I20260811 02:49:24.162145 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:49:24.162869 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 11306
I20260811 02:49:24.190989 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 11534
I20260811 02:49:24.207724 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 11757
I20260811 02:49:24.226667 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 11112
I20260811 02:49:24.240345 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:44197
--webserver_interface=127.21.236.190
--webserver_port=37117
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:44197
--ranger_config_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.236.148:51983/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.236.148:51983/kms with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:24.360181 12007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:24.360459 12007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:24.360498 12007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:24.364625 12007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:49:24.364691 12007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:24.364710 12007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:24.364727 12007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:49:24.364771 12007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:49:24.370186 12007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.236.148:51983/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:44197
--ranger_config_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:44197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=37117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:24.371253 12007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:24.372279 12007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:24.381418 12020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:24.381471 12030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:24.381369 12024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:24.381685 12007 server_base.cc:1034] running on GCE node
I20260811 02:49:24.382247 12007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:24.383453 12007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:24.384632 12007 hybrid_clock.cc:648] HybridClock initialized: now 1786416564384611 us; error 32 us; skew 500 ppm
Aug 11 02:49:24 dist-test-slave-n326 krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416564, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:24.387642 12007 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:49:24.388664 12007 webserver.cc:460] Webserver started at http://127.21.236.190:37117/ using document root <none> and password file <none>
I20260811 02:49:24.389253 12007 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:24.389361 12007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 02:49:24 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416564, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:49:24.470768 12007 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:49:24.473347 12042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:24.474804 12007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 02:49:24.474962 12007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "86e9b70cb78a4eac86634f14b9a7f619"
format_stamp: "Formatted at 2026-08-11 02:49:22 on dist-test-slave-n326"
server_key: "431fc6d187d3664fbea2dc72956f2d39"
server_key_iv: "1ea66dfe61bc71e5bfab5b8817a4737a"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:24.475607 12007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:24.502414 12007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:24.505530 12007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:24.505796 12007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:24.513480 12007 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:44197
I20260811 02:49:24.513507 12138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:44197 every 8 connection(s)
I20260811 02:49:24.514658 12007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:49:24.519395 12149 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:49:24.519644 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 12007
I20260811 02:49:24.520129 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.236.148:51983/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:41127
--local_ip_for_outbound_sockets=127.21.236.129
--tserver_master_addrs=127.21.236.190:44197
--webserver_port=39039
--webserver_interface=127.21.236.129
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.236.148:51983/kms with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:49:24.521433 12149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619: Bootstrap starting.
I20260811 02:49:24.539345 12149 log.cc:824] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:24.546573 12149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:49:24.547043 12149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619: Bootstrap complete.
I20260811 02:49:24.551741 12149 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e9b70cb78a4eac86634f14b9a7f619" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 44197 } }
I20260811 02:49:24.552210 12149 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86e9b70cb78a4eac86634f14b9a7f619, State: Initialized, Role: FOLLOWER
I20260811 02:49:24.552743 12149 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e9b70cb78a4eac86634f14b9a7f619" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 44197 } }
I20260811 02:49:24.552917 12149 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:49:24.553007 12149 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:49:24.553140 12149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:49:24.555297 12149 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e9b70cb78a4eac86634f14b9a7f619" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 44197 } }
I20260811 02:49:24.555656 12149 leader_election.cc:302] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86e9b70cb78a4eac86634f14b9a7f619; no voters:
I20260811 02:49:24.556003 12149 leader_election.cc:288] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:49:24.556155 12159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:49:24.556414 12159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [term 2 LEADER]: Becoming Leader. State: Replica: 86e9b70cb78a4eac86634f14b9a7f619, State: Running, Role: LEADER
I20260811 02:49:24.556756 12159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e9b70cb78a4eac86634f14b9a7f619" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 44197 } }
I20260811 02:49:24.557307 12149 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:49:24.558295 12160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "86e9b70cb78a4eac86634f14b9a7f619" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e9b70cb78a4eac86634f14b9a7f619" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 44197 } } }
I20260811 02:49:24.558470 12160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:24.558867 12161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86e9b70cb78a4eac86634f14b9a7f619. Latest consensus state: current_term: 2 leader_uuid: "86e9b70cb78a4eac86634f14b9a7f619" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86e9b70cb78a4eac86634f14b9a7f619" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 44197 } } }
I20260811 02:49:24.559033 12161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:24.559425 12176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:49:24.560067 12149 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 02:49:24.566308 12176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:49:24.567242 12176 catalog_manager.cc:1259] Loaded cluster ID: d8695b63c401464a901831321dbbd0df
I20260811 02:49:24.567358 12176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:49:24.570624 12176 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:49:24.571485 12176 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619: Loaded TSK: 0
I20260811 02:49:24.572913 12176 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 02:49:24.647351 12154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:24.647768 12154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:24.647838 12154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:24.652633 12154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:24.652719 12154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:24.652813 12154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:49:24.658830 12154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.236.148:51983/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:41127
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=39039
--tserver_master_addrs=127.21.236.190:44197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:24.660391 12154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:24.661494 12154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:24.671888 12224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:24.671903 12219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:24.671931 12218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:24.672946 12154 server_base.cc:1034] running on GCE node
I20260811 02:49:24.673401 12154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:24.674134 12154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:24.675326 12154 hybrid_clock.cc:648] HybridClock initialized: now 1786416564675285 us; error 50 us; skew 500 ppm
Aug 11 02:49:24 dist-test-slave-n326 krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416564, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:24.679111 12154 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:49:24.680452 12154 webserver.cc:460] Webserver started at http://127.21.236.129:39039/ using document root <none> and password file <none>
I20260811 02:49:24.681313 12154 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:24.681535 12154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 02:49:24 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416564, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:49:24.722059 12154 fs_manager.cc:709] Time spent opening directory manager: real 0.016s user 0.003s sys 0.001s
I20260811 02:49:24.726377 12235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:24.728531 12154 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
I20260811 02:49:24.728773 12154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "d595b681c94a4c3cb88a3de3740f3881"
format_stamp: "Formatted at 2026-08-11 02:49:23 on dist-test-slave-n326"
server_key: "d93a0ff2115b16821eeaa300f8f52e58"
server_key_iv: "49e6365175fa06566a00d2b2c3e2c6a1"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:24.729734 12154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:24.749408 12154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:24.753548 12154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:24.753789 12154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:24.754599 12154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:24.755790 12154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:24.755873 12154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:24.755939 12154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:24.755971 12154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:24.769194 12154 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:41127
I20260811 02:49:24.770831 12154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:49:24.771498 12439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:41127 every 8 connection(s)
I20260811 02:49:24.779397 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 12154
I20260811 02:49:24.779966 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.236.148:51983/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:41343
--local_ip_for_outbound_sockets=127.21.236.130
--tserver_master_addrs=127.21.236.190:44197
--webserver_port=37607
--webserver_interface=127.21.236.130
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.236.148:51983/kms with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:49:24 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416564, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:24.798179 12445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:24.774729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:44197 (local address 127.21.236.129:57493)
0811 02:49:24.776264 (+ 1535us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:24.776323 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:24.777366 (+ 1043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:24.779506 (+ 2140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:24.779514 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:24.779898 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:24.780517 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:24.780536 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:24.782944 (+ 2408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:24.782948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:24.784640 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:24.784656 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:24.786128 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:24.786132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:24.786349 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:24.788661 (+ 2312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:24.788686 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:24.790699 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:24.793624 (+ 2925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:24.793632 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:24.793651 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:24.794044 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:24.794586 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:24.794592 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:24.794594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:24.794808 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:24.795805 (+ 997us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:24.795813 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:24.796164 (+ 351us) client_negotiation.cc:769] Sending connection context
0811 02:49:24.796401 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 02:49:24.796837 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1310,"thread_start_us":212,"threads_started":1}
I20260811 02:49:24.800529 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:24.774617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:57493 (local address 127.21.236.190:44197)
0811 02:49:24.777856 (+ 3239us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:24.777878 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:24.777919 (+ 41us) server_negotiation.cc:407] Connection header received
0811 02:49:24.778206 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:24.778219 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:24.778594 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:24.778958 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:24.780711 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:24.782737 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:24.784823 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:24.785637 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:24.785827 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:24.790999 (+ 5172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:24.791007 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:24.791029 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:24.791120 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:24.793449 (+ 2329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:24.794217 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:24.794224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:24.794229 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:24.794429 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:24.794931 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:24.794935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:24.794936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:24.795575 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:24.795741 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:24.799048 (+ 3307us) server_negotiation.cc:299] Negotiation successful
0811 02:49:24.799465 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3076,"thread_start_us":140,"threads_started":1}
I20260811 02:49:24.803627 12442 heartbeater.cc:347] Connected to a master server at 127.21.236.190:44197
I20260811 02:49:24.803900 12442 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:24.804535 12442 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
W20260811 02:49:24.807242 12442 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:44197 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 02:49:24.807384 12442 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:24.807582 12442 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
I20260811 02:49:24.808533 12442 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:24.808754 12442 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
W20260811 02:49:24.809576 12442 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:49:24.955804 12446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:24.956111 12446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:24.956187 12446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:24.960979 12446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:24.961072 12446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:24.961166 12446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:49:24.966850 12446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.236.148:51983/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:41343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=37607
--tserver_master_addrs=127.21.236.190:44197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:24.968122 12446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:24.969149 12446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:24.979874 12464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:24.979874 12460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:24.980183 12461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:24.980319 12446 server_base.cc:1034] running on GCE node
I20260811 02:49:24.981405 12446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:24.982219 12446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:24.983431 12446 hybrid_clock.cc:648] HybridClock initialized: now 1786416564983379 us; error 71 us; skew 500 ppm
Aug 11 02:49:24 dist-test-slave-n326 krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416564, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:24.987599 12446 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:49:24.988904 12446 webserver.cc:460] Webserver started at http://127.21.236.130:37607/ using document root <none> and password file <none>
I20260811 02:49:24.989728 12446 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:24.989861 12446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 02:49:24 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416564, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:49:25.015375 12446 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.004s sys 0.000s
I20260811 02:49:25.020615 12478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:25.022595 12446 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260811 02:49:25.022753 12446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "34986516f5844695954d9ecab88fa881"
format_stamp: "Formatted at 2026-08-11 02:49:23 on dist-test-slave-n326"
server_key: "6fa3a7154391115d24983dc5a81fc670"
server_key_iv: "a2789d79baed4f5a1b681e2eaeb9071b"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:25.023602 12446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:25.042807 12446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:25.046741 12446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:25.046980 12446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:25.047837 12446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:25.049069 12446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:25.049144 12446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:25.049211 12446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:25.049266 12446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:25.063054 12446 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:41343
I20260811 02:49:25.063305 12702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:41343 every 8 connection(s)
I20260811 02:49:25.065536 12446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:49:25.067418 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 12446
I20260811 02:49:25.067970 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.236.148:51983/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:35223
--local_ip_for_outbound_sockets=127.21.236.131
--tserver_master_addrs=127.21.236.190:44197
--webserver_port=36087
--webserver_interface=127.21.236.131
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.236.148:51983/kms with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:49:25 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416564, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:25.085086 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:25.068019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:42451 (local address 127.21.236.190:44197)
0811 02:49:25.068175 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:25.068179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:25.069317 (+ 1138us) server_negotiation.cc:407] Connection header received
0811 02:49:25.070908 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:25.070911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:25.070987 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:25.071115 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:25.072759 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:25.073604 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:25.075375 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:25.075718 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:25.075798 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:25.080218 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:25.080222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:25.080225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:25.080275 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:25.082246 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:25.082959 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:25.082963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:25.082965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:25.083116 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:25.083620 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:25.083624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:25.083626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:25.083872 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:25.083970 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:25.084738 (+ 768us) server_negotiation.cc:299] Negotiation successful
0811 02:49:25.084879 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 02:49:25.087540 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:25.068394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:44197 (local address 127.21.236.130:42451)
0811 02:49:25.069092 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:25.069151 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:25.070575 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:25.071363 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:25.071376 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:25.071799 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:25.072565 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:25.072590 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:25.073774 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:25.073778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:25.075228 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:25.075240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:25.076027 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:25.076033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:25.076235 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:25.076928 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:25.076960 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:25.079992 (+ 3032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:25.082409 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:25.082415 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:25.082435 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:25.082816 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:25.083280 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:25.083282 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:25.083284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:25.083492 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:25.084066 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:25.084073 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:25.084424 (+ 351us) client_negotiation.cc:769] Sending connection context
0811 02:49:25.085591 (+ 1167us) client_negotiation.cc:240] Negotiation successful
0811 02:49:25.085989 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":153,"threads_started":1}
I20260811 02:49:25.089668 12704 heartbeater.cc:347] Connected to a master server at 127.21.236.190:44197
I20260811 02:49:25.089927 12704 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:25.090582 12704 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
W20260811 02:49:25.092500 12704 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:44197 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 02:49:25.092638 12704 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:25.092831 12704 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
I20260811 02:49:25.093784 12704 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:25.093956 12704 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
W20260811 02:49:25.094841 12704 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:49:25.232538 12709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:25.232818 12709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:25.232862 12709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:25.237628 12709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:25.237694 12709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:25.237782 12709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:49:25.244732 12709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.236.148:51983/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:35223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=36087
--tserver_master_addrs=127.21.236.190:44197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:25.246163 12709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:25.247640 12709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:25.258313 12718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:25.258646 12717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:25.258313 12720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:25.259624 12709 server_base.cc:1034] running on GCE node
I20260811 02:49:25.260174 12709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:25.260892 12709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:25.268273 12709 hybrid_clock.cc:648] HybridClock initialized: now 1786416565268191 us; error 94 us; skew 500 ppm
Aug 11 02:49:25 dist-test-slave-n326 krb5kdc[10359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:25.271775 12709 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:49:25.272857 12709 webserver.cc:460] Webserver started at http://127.21.236.131:36087/ using document root <none> and password file <none>
I20260811 02:49:25.273469 12709 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:25.273593 12709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 02:49:25 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:49:25.298492 12709 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.000s
I20260811 02:49:25.301484 12732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:25.303288 12709 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 02:49:25.303440 12709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "263218133f274ab8a56a9098e559edaa"
format_stamp: "Formatted at 2026-08-11 02:49:24 on dist-test-slave-n326"
server_key: "00de4d7456b815990eaf0b8ee591cf4c"
server_key_iv: "76530a2101ae7e341407a5c5105bdf19"
server_key_version: "kuduclusterkey@0"
I20260811 02:49:25.304193 12709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:25.323731 12709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:25.327587 12709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:25.327822 12709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:25.328608 12709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:25.329891 12709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:25.329967 12709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:25.330034 12709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:25.330066 12709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:25.348749 12709 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:35223
I20260811 02:49:25.348832 12939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:35223 every 8 connection(s)
I20260811 02:49:25.350109 12709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:49:25.360425 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 12709
Aug 11 02:49:25 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:25.368531 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:25.352248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:55959 (local address 127.21.236.190:44197)
0811 02:49:25.352394 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:25.352399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:25.353364 (+ 965us) server_negotiation.cc:407] Connection header received
0811 02:49:25.354600 (+ 1236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:25.354604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:25.354659 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:25.354710 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:25.356349 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:25.357165 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:25.359037 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:25.359417 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:25.359502 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:25.363990 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:25.363993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:25.363996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:25.364025 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:25.365799 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:25.366466 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:25.366470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:25.366472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:25.366562 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:25.366984 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:25.366988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:25.366990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:25.367242 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:25.367318 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:25.368228 (+ 910us) server_negotiation.cc:299] Negotiation successful
0811 02:49:25.368364 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 02:49:25.371961 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:25.352713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:44197 (local address 127.21.236.131:55959)
0811 02:49:25.353208 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:25.353261 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:25.354342 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:25.355150 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:25.355160 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:25.355505 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:25.356174 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:25.356195 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:25.357333 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:25.357336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:25.358879 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:25.358891 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:25.359750 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:25.359755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:25.359963 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:25.361953 (+ 1990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:25.361974 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:25.363775 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:25.365964 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:25.365971 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:25.366002 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:25.366343 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:25.366676 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:25.366679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:25.366681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:25.366860 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:25.367450 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:25.367458 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:25.367937 (+ 479us) client_negotiation.cc:769] Sending connection context
0811 02:49:25.369340 (+ 1403us) client_negotiation.cc:240] Negotiation successful
0811 02:49:25.369697 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":161,"threads_started":1}
I20260811 02:49:25.373451 12942 heartbeater.cc:347] Connected to a master server at 127.21.236.190:44197
I20260811 02:49:25.373711 12942 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:25.374437 12942 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
W20260811 02:49:25.377655 12942 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:44197 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 02:49:25.377857 12942 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:25.378116 12942 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
I20260811 02:49:25.379210 12942 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:25.379401 12942 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
I20260811 02:49:25.381004 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:25.361499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34310 (local address 127.21.236.190:44197)
0811 02:49:25.368721 (+ 7222us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:25.368725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:25.368739 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:49:25.368772 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:25.368775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:25.368832 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:25.368916 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:25.369904 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:25.370619 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:25.372804 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:25.373144 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:25.373249 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:25.374990 (+ 1741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:25.374994 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:25.374996 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:25.375021 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:25.378236 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:25.378762 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:25.378765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:25.378767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:25.378856 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:25.379620 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:25.379623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:25.379625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:25.379852 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:25.380088 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:25.380663 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 02:49:25.380780 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7160,"thread_start_us":106,"threads_started":1}
W20260811 02:49:25.381476 12942 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 02:49:25 dist-test-slave-n326 krb5kdc[10359](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786416565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:25.809842 12442 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:25.810218 12442 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
I20260811 02:49:25.811605 12077 ts_manager.cc:194] Registered new tserver with Master: d595b681c94a4c3cb88a3de3740f3881 (127.21.236.129:41127)
I20260811 02:49:25.813537 12077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:57493
I20260811 02:49:26.095170 12704 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:26.095535 12704 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
I20260811 02:49:26.096462 12076 ts_manager.cc:194] Registered new tserver with Master: 34986516f5844695954d9ecab88fa881 (127.21.236.130:41343)
I20260811 02:49:26.097381 12076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:42451
Aug 11 02:49:26 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
I20260811 02:49:26.381708 12942 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:26.382033 12942 heartbeater.cc:511] Master 127.21.236.190:44197 requested a full tablet report, sending...
I20260811 02:49:26.382866 12076 ts_manager.cc:194] Registered new tserver with Master: 263218133f274ab8a56a9098e559edaa (127.21.236.131:35223)
I20260811 02:49:26.383694 12076 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:55959
I20260811 02:49:26.394470 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:49:26.407003 12977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:26.396447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60208 (local address 127.21.236.190:44197)
0811 02:49:26.396766 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:26.396771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:26.396789 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:49:26.396901 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:26.396905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:26.396962 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:26.397076 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:26.397964 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.398923 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.400106 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.400570 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.400700 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:26.402668 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:26.402672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:26.402676 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:26.402708 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:26.404953 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:26.405496 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:26.405500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:26.405502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:26.405616 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:26.405934 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:26.405938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:26.405940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:26.406212 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:26.406371 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:26.406637 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 02:49:26.406788 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":132,"threads_started":1}
I20260811 02:49:26.412021 12076 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60208:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:49:26.417521 12076 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:49:26.440037 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:26.430658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:41127 (local address 127.0.0.1:40390)
0811 02:49:26.433718 (+ 3060us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:26.433778 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:26.433914 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:26.435166 (+ 1252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:26.435173 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:26.435197 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:26.435340 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:26.435347 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.436810 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.436813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:26.438156 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:26.438168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.439477 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.439481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:26.439590 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:26.439639 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 02:49:26.439757 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 02:49:26.439883 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2855,"thread_start_us":121,"threads_started":1}
I20260811 02:49:26.440716 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:26.431563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40390 (local address 127.21.236.129:41127)
0811 02:49:26.434465 (+ 2902us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:26.434473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:26.434498 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:49:26.434577 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:26.434582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:26.434818 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:26.434962 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:26.435510 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.436670 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.438302 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.439153 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.439273 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:26.439413 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:26.440437 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0811 02:49:26.440571 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2789,"thread_start_us":144,"threads_started":1}
I20260811 02:49:26.444353 12303 tablet_service.cc:1467] Processing CreateTablet for tablet 95ecdbf03b8e42f1a578d3a679fb616e (DEFAULT_TABLE table=test-table [id=f87905bdd7f5413eb1906844ea47033a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:49:26.446591 12992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:26.432442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:35223 (local address 127.0.0.1:36406)
0811 02:49:26.434142 (+ 1700us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:26.434155 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:26.434228 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:26.442314 (+ 8086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:26.442318 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:26.442337 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:26.442426 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:26.442432 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.443944 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.443948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:26.445057 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:26.445069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.446174 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.446178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:26.446271 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:26.446302 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 02:49:26.446351 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 02:49:26.446445 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1638,"thread_start_us":91,"threads_started":1}
I20260811 02:49:26.447146 12303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95ecdbf03b8e42f1a578d3a679fb616e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:26.447342 12994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:26.434344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36406 (local address 127.21.236.131:35223)
0811 02:49:26.441824 (+ 7480us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:26.441832 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:26.441863 (+ 31us) server_negotiation.cc:407] Connection header received
0811 02:49:26.441943 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:26.441948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:26.442161 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:26.442578 (+ 417us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:26.442727 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.443810 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.445228 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.446031 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.446899 (+ 868us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:26.447034 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:26.447111 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 02:49:26.447208 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7361,"thread_start_us":179,"threads_started":1}
I20260811 02:49:26.449292 12987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:26.433130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:41343 (local address 127.0.0.1:42848)
0811 02:49:26.434717 (+ 1587us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:26.434734 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:26.434842 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:26.443801 (+ 8959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:26.443804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:26.443827 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:26.443925 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:26.443932 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.446461 (+ 2529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.446467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:26.447747 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:26.447759 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.448793 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.448797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:26.448903 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:26.448933 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 02:49:26.449036 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 02:49:26.449114 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1513,"thread_start_us":90,"threads_started":1}
I20260811 02:49:26.450424 12832 tablet_service.cc:1467] Processing CreateTablet for tablet 95ecdbf03b8e42f1a578d3a679fb616e (DEFAULT_TABLE table=test-table [id=f87905bdd7f5413eb1906844ea47033a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:49:26.451660 12832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95ecdbf03b8e42f1a578d3a679fb616e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:26.455487 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:26.433306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42848 (local address 127.21.236.130:41343)
0811 02:49:26.443375 (+ 10069us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:26.443381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:26.443406 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:49:26.443480 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:26.443485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:26.443665 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:26.443810 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:26.445252 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.446353 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.447873 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.448668 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.448807 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:26.448932 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:26.455177 (+ 6245us) server_negotiation.cc:299] Negotiation successful
0811 02:49:26.455323 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9950,"thread_start_us":57,"threads_started":1}
I20260811 02:49:26.458846 12566 tablet_service.cc:1467] Processing CreateTablet for tablet 95ecdbf03b8e42f1a578d3a679fb616e (DEFAULT_TABLE table=test-table [id=f87905bdd7f5413eb1906844ea47033a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:49:26.460047 12566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95ecdbf03b8e42f1a578d3a679fb616e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:26.462055 12995 tablet_bootstrap.cc:492] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa: Bootstrap starting.
I20260811 02:49:26.464378 12995 tablet_bootstrap.cc:654] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:26.465265 12995 log.cc:824] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:26.467911 12995 tablet_bootstrap.cc:492] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa: No bootstrap required, opened a new log
I20260811 02:49:26.468129 12995 ts_tablet_manager.cc:1397] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:49:26.472374 12995 raft_consensus.cc:348] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595b681c94a4c3cb88a3de3740f3881" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41127 } } peers { permanent_uuid: "263218133f274ab8a56a9098e559edaa" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 35223 } } peers { permanent_uuid: "34986516f5844695954d9ecab88fa881" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41343 } }
I20260811 02:49:26.472679 12995 raft_consensus.cc:374] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:26.472730 12995 raft_consensus.cc:729] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 263218133f274ab8a56a9098e559edaa, State: Initialized, Role: FOLLOWER
I20260811 02:49:26.473184 12995 consensus_queue.cc:260] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595b681c94a4c3cb88a3de3740f3881" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41127 } } peers { permanent_uuid: "263218133f274ab8a56a9098e559edaa" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 35223 } } peers { permanent_uuid: "34986516f5844695954d9ecab88fa881" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41343 } }
I20260811 02:49:26.474259 12995 ts_tablet_manager.cc:1428] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:49:26.474659 12942 heartbeater.cc:503] Master 127.21.236.190:44197 was elected leader, sending a full tablet report...
I20260811 02:49:26.475087 12999 tablet_bootstrap.cc:492] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881: Bootstrap starting.
I20260811 02:49:26.477521 12999 tablet_bootstrap.cc:654] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:26.478478 12999 log.cc:824] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:26.481747 13002 tablet_bootstrap.cc:492] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881: Bootstrap starting.
I20260811 02:49:26.484383 13002 tablet_bootstrap.cc:654] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:26.484843 12999 tablet_bootstrap.cc:492] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881: No bootstrap required, opened a new log
I20260811 02:49:26.485069 12999 ts_tablet_manager.cc:1397] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260811 02:49:26.485531 13002 log.cc:824] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:26.488530 13002 tablet_bootstrap.cc:492] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881: No bootstrap required, opened a new log
I20260811 02:49:26.488744 13002 ts_tablet_manager.cc:1397] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 02:49:26.489074 12999 raft_consensus.cc:348] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595b681c94a4c3cb88a3de3740f3881" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41127 } } peers { permanent_uuid: "263218133f274ab8a56a9098e559edaa" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 35223 } } peers { permanent_uuid: "34986516f5844695954d9ecab88fa881" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41343 } }
I20260811 02:49:26.489355 12999 raft_consensus.cc:374] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:26.489401 12999 raft_consensus.cc:729] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34986516f5844695954d9ecab88fa881, State: Initialized, Role: FOLLOWER
I20260811 02:49:26.489882 12999 consensus_queue.cc:260] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595b681c94a4c3cb88a3de3740f3881" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41127 } } peers { permanent_uuid: "263218133f274ab8a56a9098e559edaa" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 35223 } } peers { permanent_uuid: "34986516f5844695954d9ecab88fa881" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41343 } }
I20260811 02:49:26.490967 12999 ts_tablet_manager.cc:1428] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:49:26.491454 12704 heartbeater.cc:503] Master 127.21.236.190:44197 was elected leader, sending a full tablet report...
I20260811 02:49:26.495189 13002 raft_consensus.cc:348] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595b681c94a4c3cb88a3de3740f3881" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41127 } } peers { permanent_uuid: "263218133f274ab8a56a9098e559edaa" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 35223 } } peers { permanent_uuid: "34986516f5844695954d9ecab88fa881" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41343 } }
I20260811 02:49:26.495678 13002 raft_consensus.cc:374] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:26.495791 13002 raft_consensus.cc:729] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d595b681c94a4c3cb88a3de3740f3881, State: Initialized, Role: FOLLOWER
I20260811 02:49:26.496337 13002 consensus_queue.cc:260] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595b681c94a4c3cb88a3de3740f3881" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41127 } } peers { permanent_uuid: "263218133f274ab8a56a9098e559edaa" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 35223 } } peers { permanent_uuid: "34986516f5844695954d9ecab88fa881" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41343 } }
I20260811 02:49:26.497112 12442 heartbeater.cc:503] Master 127.21.236.190:44197 was elected leader, sending a full tablet report...
I20260811 02:49:26.497509 13002 ts_tablet_manager.cc:1428] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
W20260811 02:49:26.523938 12443 tablet.cc:2369] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 02:49:26 dist-test-slave-n326 krb5kdc[10359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for HTTP/127.21.236.148@KRBTEST.COM
W20260811 02:49:26.568977 12705 tablet.cc:2369] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:49:26.575879 13007 raft_consensus.cc:482] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:49:26.576156 13007 raft_consensus.cc:504] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595b681c94a4c3cb88a3de3740f3881" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41127 } } peers { permanent_uuid: "263218133f274ab8a56a9098e559edaa" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 35223 } } peers { permanent_uuid: "34986516f5844695954d9ecab88fa881" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41343 } }
I20260811 02:49:26.577651 13007 leader_election.cc:288] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 263218133f274ab8a56a9098e559edaa (127.21.236.131:35223), 34986516f5844695954d9ecab88fa881 (127.21.236.130:41343)
I20260811 02:49:26.588848 12993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:26.580765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:57027 (local address 127.21.236.130:41343)
0811 02:49:26.580902 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:26.580908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:26.583407 (+ 2499us) server_negotiation.cc:407] Connection header received
0811 02:49:26.583644 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:26.583648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:26.583701 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:26.583752 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:26.584225 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.585128 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.587382 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.588032 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.588150 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:26.588203 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:26.588605 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 02:49:26.588697 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 02:49:26.589370 13012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:26.578516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:41343 (local address 127.21.236.129:57027)
0811 02:49:26.583303 (+ 4787us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:26.583320 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:26.583489 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:26.583894 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:26.583898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:26.583917 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:26.584074 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:26.584082 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.585511 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.585515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:26.587223 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:26.587236 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.588307 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.588311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:26.588436 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:26.588460 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:49:26.589091 (+ 631us) client_negotiation.cc:240] Negotiation successful
0811 02:49:26.589179 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4708,"thread_start_us":140,"threads_started":1}
I20260811 02:49:26.590086 12571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ecdbf03b8e42f1a578d3a679fb616e" candidate_uuid: "d595b681c94a4c3cb88a3de3740f3881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34986516f5844695954d9ecab88fa881" is_pre_election: true
I20260811 02:49:26.590384 12571 raft_consensus.cc:2454] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d595b681c94a4c3cb88a3de3740f3881 in term 0.
I20260811 02:49:26.590919 12241 leader_election.cc:302] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34986516f5844695954d9ecab88fa881, d595b681c94a4c3cb88a3de3740f3881; no voters:
I20260811 02:49:26.591259 13007 raft_consensus.cc:2781] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:49:26.591364 13007 raft_consensus.cc:482] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:49:26.591452 13007 raft_consensus.cc:3037] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:26.592757 13007 raft_consensus.cc:504] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595b681c94a4c3cb88a3de3740f3881" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41127 } } peers { permanent_uuid: "263218133f274ab8a56a9098e559edaa" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 35223 } } peers { permanent_uuid: "34986516f5844695954d9ecab88fa881" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41343 } }
I20260811 02:49:26.593217 13007 leader_election.cc:288] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [CANDIDATE]: Term 1 election: Requested vote from peers 263218133f274ab8a56a9098e559edaa (127.21.236.131:35223), 34986516f5844695954d9ecab88fa881 (127.21.236.130:41343)
I20260811 02:49:26.593609 12571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ecdbf03b8e42f1a578d3a679fb616e" candidate_uuid: "d595b681c94a4c3cb88a3de3740f3881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34986516f5844695954d9ecab88fa881"
I20260811 02:49:26.593760 12571 raft_consensus.cc:3037] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:26.595012 12571 raft_consensus.cc:2454] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595b681c94a4c3cb88a3de3740f3881 in term 1.
I20260811 02:49:26.595412 12241 leader_election.cc:302] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34986516f5844695954d9ecab88fa881, d595b681c94a4c3cb88a3de3740f3881; no voters:
I20260811 02:49:26.595597 13007 raft_consensus.cc:2781] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:49:26.595625 12994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:26.580588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:51269 (local address 127.21.236.131:35223)
0811 02:49:26.580732 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:26.580737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:26.580752 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:49:26.580810 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:26.580813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:26.580867 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:26.580948 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:26.581849 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.582636 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.585199 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.585818 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.585919 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:26.585974 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:26.595325 (+ 9351us) server_negotiation.cc:299] Negotiation successful
0811 02:49:26.595447 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 02:49:26.595846 13007 raft_consensus.cc:686] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [term 1 LEADER]: Becoming Leader. State: Replica: d595b681c94a4c3cb88a3de3740f3881, State: Running, Role: LEADER
I20260811 02:49:26.596177 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:26.578068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:35223 (local address 127.21.236.129:51269)
0811 02:49:26.578956 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:26.578993 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:26.579246 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:26.581495 (+ 2249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:26.581499 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:26.581523 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:26.581701 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:26.581709 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.583274 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.583277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:26.585047 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:26.585060 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.593108 (+ 8048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.593111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:26.593228 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:26.593251 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:49:26.595801 (+ 2550us) client_negotiation.cc:240] Negotiation successful
0811 02:49:26.595900 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"thread_start_us":138,"threads_started":1}
I20260811 02:49:26.596279 13007 consensus_queue.cc:237] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595b681c94a4c3cb88a3de3740f3881" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41127 } } peers { permanent_uuid: "263218133f274ab8a56a9098e559edaa" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 35223 } } peers { permanent_uuid: "34986516f5844695954d9ecab88fa881" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41343 } }
I20260811 02:49:26.597817 12863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ecdbf03b8e42f1a578d3a679fb616e" candidate_uuid: "d595b681c94a4c3cb88a3de3740f3881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263218133f274ab8a56a9098e559edaa" is_pre_election: true
I20260811 02:49:26.598156 12863 raft_consensus.cc:2454] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d595b681c94a4c3cb88a3de3740f3881 in term 0.
I20260811 02:49:26.598546 12861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ecdbf03b8e42f1a578d3a679fb616e" candidate_uuid: "d595b681c94a4c3cb88a3de3740f3881" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263218133f274ab8a56a9098e559edaa"
I20260811 02:49:26.598671 12861 raft_consensus.cc:3037] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:26.600096 12861 raft_consensus.cc:2454] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d595b681c94a4c3cb88a3de3740f3881 in term 1.
W20260811 02:49:26.602921 12943 tablet.cc:2369] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:49:26.603145 12076 catalog_manager.cc:5495] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 reported cstate change: term changed from 0 to 1, leader changed from <none> to d595b681c94a4c3cb88a3de3740f3881 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "d595b681c94a4c3cb88a3de3740f3881" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d595b681c94a4c3cb88a3de3740f3881" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "263218133f274ab8a56a9098e559edaa" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 35223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34986516f5844695954d9ecab88fa881" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41343 } health_report { overall_health: UNKNOWN } } }
I20260811 02:49:26.646588 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:26.641468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40402 (local address 127.21.236.129:41127)
0811 02:49:26.641587 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:26.641592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:26.641615 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:49:26.641704 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:26.641707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:26.641770 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:26.641888 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:49:26.642303 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.643105 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.644290 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:26.644656 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:26.644764 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:26.645202 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:49:26.645809 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:49:26.645939 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:26.646358 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 02:49:26.646421 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:49:26.654752 12571 raft_consensus.cc:1261] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881 [term 1 FOLLOWER]: Refusing update from remote peer d595b681c94a4c3cb88a3de3740f3881: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:49:26.655903 12861 raft_consensus.cc:1261] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa [term 1 FOLLOWER]: Refusing update from remote peer d595b681c94a4c3cb88a3de3740f3881: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:49:26.656424 13015 consensus_queue.cc:1035] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34986516f5844695954d9ecab88fa881" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:49:26.656616 13007 consensus_queue.cc:1035] T 95ecdbf03b8e42f1a578d3a679fb616e P d595b681c94a4c3cb88a3de3740f3881 [LEADER]: Connected to new peer: Peer: permanent_uuid: "263218133f274ab8a56a9098e559edaa" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:49:26.685895 13021 mvcc.cc:205] Tried to move back new op lower bound from 7317162257009422336 to 7317162256794935296. Current Snapshot: MvccSnapshot[applied={T|T < 7317162257009422336}]
I20260811 02:49:26.698530 12076 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:49:26.698733 12076 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:49:26.701948 12076 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86e9b70cb78a4eac86634f14b9a7f619: Sending DeleteTablet for 3 replicas of tablet 95ecdbf03b8e42f1a578d3a679fb616e
I20260811 02:49:26.703245 12303 tablet_service.cc:1514] Processing DeleteTablet for tablet 95ecdbf03b8e42f1a578d3a679fb616e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:49:26 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:40390
I20260811 02:49:26.703863 12832 tablet_service.cc:1514] Processing DeleteTablet for tablet 95ecdbf03b8e42f1a578d3a679fb616e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:49:26 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:36406
I20260811 02:49:26.704566 12566 tablet_service.cc:1514] Processing DeleteTablet for tablet 95ecdbf03b8e42f1a578d3a679fb616e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:49:26 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:42848
I20260811 02:49:26.706068 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 12154
I20260811 02:49:26.710841 13037 tablet_replica.cc:330] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa: stopping tablet replica
I20260811 02:49:26.711158 13038 tablet_replica.cc:330] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881: stopping tablet replica
I20260811 02:49:26.711624 13038 raft_consensus.cc:2229] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:49:26.711966 13037 raft_consensus.cc:2229] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:49:26.712567 13038 raft_consensus.cc:2258] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:49:26.712800 13037 raft_consensus.cc:2258] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:49:26.716564 13038 ts_tablet_manager.cc:1905] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:49:26.719769 13038 ts_tablet_manager.cc:1918] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:49:26.719897 13038 log.cc:1196] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/95ecdbf03b8e42f1a578d3a679fb616e
I20260811 02:49:26.720293 13038 ts_tablet_manager.cc:1939] T 95ecdbf03b8e42f1a578d3a679fb616e P 34986516f5844695954d9ecab88fa881: Deleting consensus metadata
I20260811 02:49:26.720501 13037 ts_tablet_manager.cc:1905] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:49:26.721776 12051 catalog_manager.cc:4841] TS 34986516f5844695954d9ecab88fa881 (127.21.236.130:41343): tablet 95ecdbf03b8e42f1a578d3a679fb616e (table test-table [id=f87905bdd7f5413eb1906844ea47033a]) successfully deleted
I20260811 02:49:26.725864 13037 ts_tablet_manager.cc:1918] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:49:26.728416 13037 log.cc:1196] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/95ecdbf03b8e42f1a578d3a679fb616e
W20260811 02:49:26.733000 12051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:41127 (error 108)
I20260811 02:49:26.733287 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 12446
I20260811 02:49:26.740029 13037 ts_tablet_manager.cc:1939] T 95ecdbf03b8e42f1a578d3a679fb616e P 263218133f274ab8a56a9098e559edaa: Deleting consensus metadata
I20260811 02:49:26.758898 12051 catalog_manager.cc:4841] TS 263218133f274ab8a56a9098e559edaa (127.21.236.131:35223): tablet 95ecdbf03b8e42f1a578d3a679fb616e (table test-table [id=f87905bdd7f5413eb1906844ea47033a]) successfully deleted
I20260811 02:49:26.759449 12987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:26.758690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:41127 (local address 127.0.0.1:40406)
0811 02:49:26.759254 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:26.759351 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466}
W20260811 02:49:26.759879 12051 catalog_manager.cc:4584] TS d595b681c94a4c3cb88a3de3740f3881 (127.21.236.129:41127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95ecdbf03b8e42f1a578d3a679fb616e: Network error: Client connection negotiation failed: client connection to 127.21.236.129:41127: connect: Connection refused (error 111)
I20260811 02:49:26.765509 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 12709
I20260811 02:49:26.783311 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 12007
I20260811 02:49:26.799800 22450 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 02:49:27.144685 22450 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 02:49:27.144819 22450 mini_ranger.cc:66] Stopping Ranger...
I20260811 02:49:27.490336 22450 mini_ranger.cc:68] Stopped Ranger
2026-08-11 02:49:27.490 UTC [10407] LOG: received smart shutdown request
2026-08-11 02:49:27.494 UTC [10407] LOG: background worker "logical replication launcher" (PID 10420) exited with exit code 1
2026-08-11 02:49:27.495 UTC [10414] LOG: shutting down
2026-08-11 02:49:27.686 UTC [10407] LOG: database system is shut down
2026-08-11T02:49:27Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51818 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:49:27 dist-test-slave-n326 krb5kdc[13057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:49:27 dist-test-slave-n326 krb5kdc[13057](info): set up 2 sockets
Aug 11 02:49:27 dist-test-slave-n326 krb5kdc[13057](info): commencing operation
krb5kdc: starting...
W20260811 02:49:29.117019 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:49:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:49:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:49:29.252221 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:42981
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:42981,127.21.236.189:39135,127.21.236.188:32827
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:29.369292 13083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:29.369629 13083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:29.369705 13083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:29.373744 13083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:49:29.373842 13083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:29.373883 13083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:29.373947 13083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:49:29.373983 13083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:49:29.379585 13083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:42981,127.21.236.189:39135,127.21.236.188:32827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:42981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:29.380748 13083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:29.381842 13083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:29.390987 13093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:29.390988 13095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:29.390987 13090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:29.391233 13083 server_base.cc:1034] running on GCE node
I20260811 02:49:29.392011 13083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:29.393191 13083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:29.394399 13083 hybrid_clock.cc:648] HybridClock initialized: now 1786416569394384 us; error 28 us; skew 500 ppm
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:29.397751 13083 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:49:29.398923 13083 webserver.cc:460] Webserver started at http://127.21.236.190:38789/ using document root <none> and password file <none>
I20260811 02:49:29.399611 13083 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:29.399681 13083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:29.399930 13083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:49:29.402267 13083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "dad952994c3f47b4b79de063e986dd96"
format_stamp: "Formatted at 2026-08-11 02:49:29 on dist-test-slave-n326"
server_key: "cb72710f86e11cea13cdb976434bfade"
server_key_iv: "fd158f09b242b8baed7cd2faec346981"
server_key_version: "encryptionkey@0"
I20260811 02:49:29.402917 13083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "dad952994c3f47b4b79de063e986dd96"
format_stamp: "Formatted at 2026-08-11 02:49:29 on dist-test-slave-n326"
server_key: "cb72710f86e11cea13cdb976434bfade"
server_key_iv: "fd158f09b242b8baed7cd2faec346981"
server_key_version: "encryptionkey@0"
I20260811 02:49:29.407183 13083 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:49:29.409853 13103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:29.411445 13083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:49:29.411625 13083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "dad952994c3f47b4b79de063e986dd96"
format_stamp: "Formatted at 2026-08-11 02:49:29 on dist-test-slave-n326"
server_key: "cb72710f86e11cea13cdb976434bfade"
server_key_iv: "fd158f09b242b8baed7cd2faec346981"
server_key_version: "encryptionkey@0"
I20260811 02:49:29.411752 13083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:29.430816 13083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:29.434134 13083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:29.434368 13083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:29.442119 13083 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:42981
I20260811 02:49:29.442422 13202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:42981 every 8 connection(s)
I20260811 02:49:29.443193 13083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:49:29.446053 13203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:29.449923 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 13083
I20260811 02:49:29.450093 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:49:29.450021 13203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:49:29.450409 22450 external_mini_cluster.cc:1428] Setting key e1585b25accb36c039e7935c6961d0f4
WARNING: no policy specified for kudu/127.21.236.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.189@KRBTEST.COM" created.
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:29.469144 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.452671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:42981 (local address 127.0.0.1:43144)
0811 02:49:29.453220 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.453282 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:29.454145 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:29.454948 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:29.454953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:29.455410 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:29.455673 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.455686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.456862 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.456865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.457618 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.457628 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.458294 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.458298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.458406 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.459161 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:29.459180 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:29.463598 (+ 4418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:29.465890 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.465901 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.465914 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.466165 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.466533 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.466537 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.466539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.466662 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.467435 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:29.467443 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:29.467590 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 02:49:29.467789 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 02:49:29.468322 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":181,"thread_start_us":171,"threads_started":1}
I20260811 02:49:29.469146 13207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.452672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43144 (local address 127.21.236.190:42981)
0811 02:49:29.453287 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.453306 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.453385 (+ 79us) server_negotiation.cc:407] Connection header received
0811 02:49:29.454480 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.454488 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.454730 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.454939 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.455836 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.456730 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.457747 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.458129 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.458316 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.463787 (+ 5471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.463793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.463806 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.463829 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.465732 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.466298 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.466305 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.466306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.466432 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.466767 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.466770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.466772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.467278 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.467433 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.467914 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.468245 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":479,"thread_start_us":168,"threads_started":1}
I20260811 02:49:29.472155 13203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:49:29.472927 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.472629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:39135 (local address 127.0.0.1:42524)
0811 02:49:29.472752 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.472820 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:39135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 02:49:29.473130 13106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.236.189:39135: connect: Connection refused (error 111)
I20260811 02:49:29.473964 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.473777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:39135 (local address 127.0.0.1:42540)
0811 02:49:29.473871 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.473913 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:39135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 02:49:29.474118 13203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.189:39135: Network error: Client connection negotiation failed: client connection to 127.21.236.189:39135: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.189.keytab.
Entry for principal kudu/127.21.236.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.189.keytab.
I20260811 02:49:29.482136 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.189:39135
--webserver_interface=127.21.236.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.189.keytab
--principal=kudu/127.21.236.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:42981,127.21.236.189:39135,127.21.236.188:32827
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:49:29.523360 13203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } attempt: 1
I20260811 02:49:29.524291 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.523965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:39135 (local address 127.0.0.1:42542)
0811 02:49:29.524138 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.524212 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:39135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 02:49:29.525031 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.524804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:39135 (local address 127.0.0.1:42552)
0811 02:49:29.524914 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.524968 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:39135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 02:49:29.525157 13203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.189:39135: Network error: Client connection negotiation failed: client connection to 127.21.236.189:39135: connect: Connection refused (error 111)
I20260811 02:49:29.593322 13203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } attempt: 2
I20260811 02:49:29.594151 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.593842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:39135 (local address 127.0.0.1:42560)
0811 02:49:29.594005 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.594073 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:39135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 02:49:29.594363 13210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:29.594648 13210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:29.594762 13210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 02:49:29.594899 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.594667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:39135 (local address 127.0.0.1:42576)
0811 02:49:29.594758 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.594812 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:39135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 02:49:29.595100 13203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.189:39135: Network error: Client connection negotiation failed: client connection to 127.21.236.189:39135: connect: Connection refused (error 111)
W20260811 02:49:29.598896 13210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:49:29.598991 13210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:29.599030 13210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:29.599083 13210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:49:29.599128 13210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:49:29.604372 13210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:42981,127.21.236.189:39135,127.21.236.188:32827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.189.keytab
--principal=kudu/127.21.236.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.189:39135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.236.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:29.605422 13210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:29.606446 13210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:29.614917 13223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:29.615173 13210 server_base.cc:1034] running on GCE node
W20260811 02:49:29.614929 13220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:29.614925 13219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:29.615840 13210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:29.617101 13210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:29.618274 13210 hybrid_clock.cc:648] HybridClock initialized: now 1786416569618254 us; error 31 us; skew 500 ppm
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:29.621080 13210 init.cc:377] Logged in from keytab as kudu/127.21.236.189@KRBTEST.COM (short username kudu)
I20260811 02:49:29.621963 13210 webserver.cc:460] Webserver started at http://127.21.236.189:37167/ using document root <none> and password file <none>
I20260811 02:49:29.622483 13210 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:29.622540 13210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:29.622706 13210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:49:29.624737 13210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/data/instance:
uuid: "c86594f95ace4affa459281052e4a48c"
format_stamp: "Formatted at 2026-08-11 02:49:29 on dist-test-slave-n326"
server_key: "59b3b050e2ac304f32f3d7ddc93acdab"
server_key_iv: "f83c966b761ff5a6fbe7aaa700f1d367"
server_key_version: "encryptionkey@0"
I20260811 02:49:29.625243 13210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/wal/instance:
uuid: "c86594f95ace4affa459281052e4a48c"
format_stamp: "Formatted at 2026-08-11 02:49:29 on dist-test-slave-n326"
server_key: "59b3b050e2ac304f32f3d7ddc93acdab"
server_key_iv: "f83c966b761ff5a6fbe7aaa700f1d367"
server_key_version: "encryptionkey@0"
I20260811 02:49:29.628969 13210 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 02:49:29.631476 13233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:29.632843 13210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:49:29.632947 13210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/wal
uuid: "c86594f95ace4affa459281052e4a48c"
format_stamp: "Formatted at 2026-08-11 02:49:29 on dist-test-slave-n326"
server_key: "59b3b050e2ac304f32f3d7ddc93acdab"
server_key_iv: "f83c966b761ff5a6fbe7aaa700f1d367"
server_key_version: "encryptionkey@0"
I20260811 02:49:29.633033 13210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:29.647634 13210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:29.650687 13210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:29.650870 13210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:29.658701 13210 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.189:39135
I20260811 02:49:29.659044 13332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.189:39135 every 8 connection(s)
I20260811 02:49:29.659929 13210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/data/info.pb
I20260811 02:49:29.662849 13335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:29.666668 13335 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:49:29.669245 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 13210
I20260811 02:49:29.669384 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-1/wal/instance
I20260811 02:49:29.669721 22450 external_mini_cluster.cc:1428] Setting key 73999a7ac8861a6518d9fdf7e310e781
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.189@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.236.188@KRBTEST.COM; defaulting to no policy
I20260811 02:49:29.681589 13207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.668698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.21.236.190:42981)
0811 02:49:29.668841 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.668845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.669764 (+ 919us) server_negotiation.cc:407] Connection header received
0811 02:49:29.671176 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.671179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.671237 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.671314 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.672697 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.673231 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.674387 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.674632 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.674752 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.677696 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.677702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.677707 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.677741 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.679464 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.679977 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.679981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.679983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.680072 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.680542 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.680545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.680546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.680738 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.680823 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.681321 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.681445 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.188@KRBTEST.COM" created.
I20260811 02:49:29.682741 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.668993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:42981 (local address 127.0.0.1:43156)
0811 02:49:29.669575 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.669639 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:29.670908 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:29.671530 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:29.671542 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:29.672047 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:29.672538 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.672552 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.673383 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.673386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.674243 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.674252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.674794 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.674799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.675023 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.675728 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:29.675757 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:29.677499 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:29.679587 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.679591 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.679604 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.679861 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.680236 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.680242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.680247 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.680420 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.680835 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:29.680841 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:29.681095 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 02:49:29.681257 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 02:49:29.681652 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":132,"threads_started":1}
I20260811 02:49:29.684358 13335 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:49:29.686255 13203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } attempt: 3
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.189@KRBTEST.COM for kudu/127.21.236.189@KRBTEST.COM
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for kudu/127.21.236.189@KRBTEST.COM
I20260811 02:49:29.694929 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.684926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:39135 (local address 127.0.0.1:42580)
0811 02:49:29.685065 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.685081 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:29.685166 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:29.685877 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:29.685879 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:29.686023 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:29.686117 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.686122 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.687697 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.687700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.688288 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.688297 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.688933 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.688939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.689100 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.689693 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:29.689713 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:29.691348 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:29.693311 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.693315 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.693317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.693514 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.693838 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.693841 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.693842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.693926 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.694401 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:29.694404 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:29.694490 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 02:49:29.694610 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 02:49:29.694742 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":3}
I20260811 02:49:29.695276 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.685024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42580 (local address 127.21.236.189:39135)
0811 02:49:29.685392 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.685405 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.685431 (+ 26us) server_negotiation.cc:407] Connection header received
0811 02:49:29.685480 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.685488 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.685688 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.685822 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.686232 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.687566 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.688409 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.688778 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.688931 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.691490 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.691504 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.691515 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.691535 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.693187 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.693624 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.693630 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.693632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.693741 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.694033 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.694037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.694038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.694282 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.694429 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.694710 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.694835 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":259,"thread_start_us":155,"threads_started":1}
I20260811 02:49:29.697633 13335 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:49:29.698397 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.698104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34040)
0811 02:49:29.698219 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.698305 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 02:49:29.698601 13237 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
I20260811 02:49:29.699344 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.686698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:39135 (local address 127.0.0.1:42584)
0811 02:49:29.686904 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.686919 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:29.687007 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:29.687384 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:29.687388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:29.687555 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:29.687686 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.687694 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.688884 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.688889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.690066 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.690077 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.690703 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.690709 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.690833 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.691347 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:29.691367 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:29.695337 (+ 3970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:29.697222 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.697227 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.697229 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.697487 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.698218 (+ 731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.698221 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.698223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.698312 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.698779 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:29.698782 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:29.698910 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 02:49:29.699053 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 02:49:29.699218 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 02:49:29.699527 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.686731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.21.236.189:39135)
0811 02:49:29.687155 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.687167 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.687181 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:49:29.687217 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.687221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.687263 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.687368 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.687815 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.688749 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.690197 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.690564 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.690678 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.695522 (+ 4844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.695525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.695528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.695553 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.697118 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.697982 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.697984 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.697986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.698074 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.698420 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.698422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.698423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.698647 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.698768 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.699167 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.699345 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":144,"threads_started":1}
I20260811 02:49:29.699600 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.699327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34044)
0811 02:49:29.699460 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.699532 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 02:49:29.699967 13335 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:32827: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
I20260811 02:49:29.700201 13203 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:49:29.700920 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.700663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34052)
0811 02:49:29.700807 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.700864 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 02:49:29.701615 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.701392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34068)
0811 02:49:29.701529 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.701570 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260811 02:49:29.701792 13203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:32827: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.188.keytab.
Entry for principal kudu/127.21.236.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.188.keytab.
I20260811 02:49:29.702909 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.188:32827
--webserver_interface=127.21.236.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.188.keytab
--principal=kudu/127.21.236.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:37449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:42981,127.21.236.189:39135,127.21.236.188:32827
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:49:29.732969 13203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } attempt: 1
I20260811 02:49:29.733939 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.733605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34072)
0811 02:49:29.733782 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.733845 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 02:49:29.734781 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.734568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34082)
0811 02:49:29.734676 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.734721 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 02:49:29.734949 13203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:32827: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
I20260811 02:49:29.749281 13335 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } attempt: 1
I20260811 02:49:29.750249 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.749894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34090)
0811 02:49:29.750075 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.750149 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 02:49:29.751123 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.750858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34100)
0811 02:49:29.750977 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.751031 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 02:49:29.751317 13335 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:32827: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
I20260811 02:49:29.810141 13203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } attempt: 2
I20260811 02:49:29.811241 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.810801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34112)
0811 02:49:29.811037 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.811144 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 02:49:29.812109 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.811851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34120)
0811 02:49:29.811988 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.812047 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 02:49:29.812254 13203 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:32827: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
I20260811 02:49:29.819533 13335 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } attempt: 2
I20260811 02:49:29.820297 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.820028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34132)
0811 02:49:29.820169 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.820225 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 02:49:29.821048 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.820831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34136)
0811 02:49:29.820944 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.820995 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 02:49:29.821187 13335 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:32827: Network error: Client connection negotiation failed: client connection to 127.21.236.188:32827: connect: Connection refused (error 111)
W20260811 02:49:29.821652 13347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:29.821893 13347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:29.821971 13347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:29.826056 13347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:49:29.826113 13347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:29.826133 13347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:29.826176 13347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:49:29.826207 13347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:49:29.831640 13347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:42981,127.21.236.189:39135,127.21.236.188:32827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.188.keytab
--principal=kudu/127.21.236.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.188:32827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.236.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:29.832710 13347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:29.833766 13347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:29.842592 13356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:29.842592 13359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:29.842597 13357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:29.842911 13347 server_base.cc:1034] running on GCE node
I20260811 02:49:29.843557 13347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:29.844722 13347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:29.845918 13347 hybrid_clock.cc:648] HybridClock initialized: now 1786416569845892 us; error 41 us; skew 500 ppm
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:29.849053 13347 init.cc:377] Logged in from keytab as kudu/127.21.236.188@KRBTEST.COM (short username kudu)
I20260811 02:49:29.849910 13347 webserver.cc:460] Webserver started at http://127.21.236.188:39549/ using document root <none> and password file <none>
I20260811 02:49:29.850440 13347 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:29.850498 13347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:29.850678 13347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:49:29.852774 13347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/data/instance:
uuid: "2787bbffd1114208bed6d32125a02e8e"
format_stamp: "Formatted at 2026-08-11 02:49:29 on dist-test-slave-n326"
server_key: "34d4cd3d74fb2927fc595a2c717b624e"
server_key_iv: "5d9494c0b32bf3fb648cab9e8aecb1ec"
server_key_version: "encryptionkey@0"
I20260811 02:49:29.853294 13347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/wal/instance:
uuid: "2787bbffd1114208bed6d32125a02e8e"
format_stamp: "Formatted at 2026-08-11 02:49:29 on dist-test-slave-n326"
server_key: "34d4cd3d74fb2927fc595a2c717b624e"
server_key_iv: "5d9494c0b32bf3fb648cab9e8aecb1ec"
server_key_version: "encryptionkey@0"
I20260811 02:49:29.856977 13347 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 02:49:29.859437 13371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:29.860746 13347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:49:29.860886 13347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/wal
uuid: "2787bbffd1114208bed6d32125a02e8e"
format_stamp: "Formatted at 2026-08-11 02:49:29 on dist-test-slave-n326"
server_key: "34d4cd3d74fb2927fc595a2c717b624e"
server_key_iv: "5d9494c0b32bf3fb648cab9e8aecb1ec"
server_key_version: "encryptionkey@0"
I20260811 02:49:29.861003 13347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:29.873140 13347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:29.876144 13347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:29.876340 13347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:29.883725 13347 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.188:32827
I20260811 02:49:29.884033 13476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.188:32827 every 8 connection(s)
I20260811 02:49:29.884735 13347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/data/info.pb
I20260811 02:49:29.887825 13477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:29.890370 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 13347
I20260811 02:49:29.890516 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786416395682572-22450-0/minicluster-data/master-2/wal/instance
I20260811 02:49:29.890802 22450 external_mini_cluster.cc:1428] Setting key 1efee7175ed1030dd67370065b514864
I20260811 02:49:29.892405 13477 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.188@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.188@KRBTEST.COM
I20260811 02:49:29.905788 13207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.894292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43170 (local address 127.21.236.190:42981)
0811 02:49:29.894512 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.894518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.894985 (+ 467us) server_negotiation.cc:407] Connection header received
0811 02:49:29.895171 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.895177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.895239 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.895326 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.896143 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.896764 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.898109 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.898416 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.898522 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.901207 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.901211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.901215 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.901251 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.903191 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.903790 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.903794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.903796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.903891 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.904328 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.904332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.904333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.904756 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.904911 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.905417 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.905586 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:49:29.906641 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.892293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34150 (local address 127.21.236.188:32827)
0811 02:49:29.892715 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.892751 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.892798 (+ 47us) server_negotiation.cc:407] Connection header received
0811 02:49:29.893437 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.893453 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.893824 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.894160 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.895248 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.896414 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.897681 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.898189 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.898337 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.901768 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.901777 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.901790 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.901816 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.903792 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.904337 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.904343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.904344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.904448 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.904827 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.904832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.904836 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.905150 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.905342 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.905598 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.905745 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":137,"threads_started":1}
I20260811 02:49:29.906641 13482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.894196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:42981 (local address 127.0.0.1:43170)
0811 02:49:29.894814 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.894876 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:29.895027 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:29.895400 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:29.895405 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:29.895795 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:29.895973 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.895985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.896895 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.896899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.897968 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.897978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.898593 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.898601 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.898729 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.899275 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:29.899302 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:29.901053 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:29.903331 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.903340 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.903351 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.903682 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.903995 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.904000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.904002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.904186 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.904894 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:29.904899 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:29.905143 (+ 244us) client_negotiation.cc:769] Sending connection context
0811 02:49:29.905333 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 02:49:29.905674 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":4,"thread_start_us":140,"threads_started":1}
I20260811 02:49:29.907970 13477 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:49:29.911401 13203 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } attempt: 3
I20260811 02:49:29.912508 13335 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } attempt: 3
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.188@KRBTEST.COM for kudu/127.21.236.189@KRBTEST.COM
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:29.918226 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.908615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42598 (local address 127.21.236.189:39135)
0811 02:49:29.908773 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.908778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.908792 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:49:29.908842 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.908845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.908901 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.909006 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.909505 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.910046 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.910820 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.911141 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.911267 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.914180 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.914184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.914187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.914213 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.916119 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.916602 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.916608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.916612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.916709 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.917050 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.917056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.917059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.917438 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.917512 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.917904 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.918038 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for kudu/127.21.236.188@KRBTEST.COM
I20260811 02:49:29.918649 13482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.908481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:39135 (local address 127.0.0.1:42598)
0811 02:49:29.908618 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.908635 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:29.908701 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:29.909039 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:29.909045 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:29.909241 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:29.909373 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.909379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.910164 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.910171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.910688 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.910695 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.911264 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.911268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.911366 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.911869 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:29.911884 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:29.913994 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:29.916230 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.916233 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.916235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.916494 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.916832 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.916838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.916842 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.916936 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.917594 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:29.917597 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:29.917703 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 02:49:29.918365 (+ 662us) client_negotiation.cc:240] Negotiation successful
0811 02:49:29.918494 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 02:49:29.919346 13207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.909265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43180 (local address 127.21.236.190:42981)
0811 02:49:29.909550 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.909556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.909568 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:49:29.909601 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.909604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.909648 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.909742 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.910376 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.910916 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.911858 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.912231 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.912345 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.915178 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.915182 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.915184 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.915214 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.917355 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.917858 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.917862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.917863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.917946 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.918247 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.918251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.918253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.918469 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.918688 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.919022 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.919177 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260811 02:49:29.919423 13477 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:49:29.922956 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.912133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34160)
0811 02:49:29.912334 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.912349 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:29.912427 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:29.913397 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:29.913400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:29.913583 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:29.913684 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.913692 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.914905 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.914910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.915933 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.915948 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.916662 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.916666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.916757 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.917348 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:29.917371 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:29.919354 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:29.921396 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.921401 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.921404 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.921645 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.921971 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.921975 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.921977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.922065 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.922485 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:29.922489 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:29.922598 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 02:49:29.922680 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 02:49:29.922804 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.189@KRBTEST.COM for kudu/127.21.236.188@KRBTEST.COM
I20260811 02:49:29.923511 13481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.912198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34160 (local address 127.21.236.188:32827)
0811 02:49:29.913200 (+ 1002us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.913204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.913214 (+ 10us) server_negotiation.cc:407] Connection header received
0811 02:49:29.913245 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.913247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.913280 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.913387 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.913832 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.914752 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.916130 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.916518 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.916996 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.919514 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.919518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.919520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.919545 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.921234 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.921778 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.921781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.921783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.921865 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.922194 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.922197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.922199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.922383 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.923186 (+ 803us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.923262 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.923371 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":932}
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.188@KRBTEST.COM for kudu/127.21.236.188@KRBTEST.COM
I20260811 02:49:29.927189 13203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96: Bootstrap starting.
I20260811 02:49:29.928135 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.912919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34176)
0811 02:49:29.913228 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.913244 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:29.913326 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:29.914112 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:29.914115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:29.914472 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:29.914585 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.914596 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.917499 (+ 2903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.917506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.918492 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.918507 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.921239 (+ 2732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.921244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.921352 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.921892 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:29.921908 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:29.923551 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:29.926183 (+ 2632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.926187 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.926189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.926439 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.926774 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.926778 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.926780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.926878 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.927601 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:29.927604 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:29.927705 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 02:49:29.927842 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 02:49:29.927973 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207}
I20260811 02:49:29.928732 13485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.913487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34176 (local address 127.21.236.188:32827)
0811 02:49:29.913861 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.913873 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.913887 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:49:29.913919 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.913922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.913962 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.914333 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.914747 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.917337 (+ 2590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.918729 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.921095 (+ 2366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.923691 (+ 2596us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.923797 (+ 106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.923800 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.923803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.923820 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.926054 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.926557 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.926562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.926564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.926667 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.926989 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.926996 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.926998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.927275 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.927404 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.928344 (+ 940us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.928484 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":137,"threads_started":1}
I20260811 02:49:29.929997 13203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:29.930770 13203 log.cc:824] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:29.931692 13335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c: Bootstrap starting.
I20260811 02:49:29.932417 13482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.920163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:32827 (local address 127.0.0.1:34192)
0811 02:49:29.920265 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:29.920280 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:29.920354 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:29.921583 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:29.921587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:29.921785 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:29.921909 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.921917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.923233 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.923240 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.924104 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:29.924117 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.924954 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.924958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:29.925041 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.925439 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:29.925455 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:29.926901 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:29.928980 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.928983 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.928985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.929242 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.931371 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:29.931374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:29.931376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:29.931468 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:29.931946 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:29.931949 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:29.932034 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 02:49:29.932141 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 02:49:29.932271 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 02:49:29.932945 13203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96: No bootstrap required, opened a new log
I20260811 02:49:29.933055 13486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.920603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.21.236.188:32827)
0811 02:49:29.921214 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.921223 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.921236 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:49:29.921268 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.921270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.921311 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.921418 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.922047 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.922956 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.924404 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.924814 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.924944 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.927059 (+ 2115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.927062 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.927065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.927105 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.928849 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.931148 (+ 2299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.931155 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.931159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.931255 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.931575 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.931578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.931579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.931788 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.931908 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.932724 (+ 816us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.932838 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":98,"threads_started":1}
I20260811 02:49:29.934614 13335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:29.935659 13335 log.cc:824] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:29.936110 13477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e: Bootstrap starting.
I20260811 02:49:29.936401 13203 raft_consensus.cc:348] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } }
I20260811 02:49:29.936662 13203 raft_consensus.cc:374] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:29.936731 13203 raft_consensus.cc:729] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad952994c3f47b4b79de063e986dd96, State: Initialized, Role: FOLLOWER
I20260811 02:49:29.937199 13203 consensus_queue.cc:260] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } }
I20260811 02:49:29.937707 13494 sys_catalog.cc:455] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } } }
I20260811 02:49:29.937903 13494 sys_catalog.cc:458] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:49:29.938232 13335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c: No bootstrap required, opened a new log
I20260811 02:49:29.938536 13203 sys_catalog.cc:564] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:49:29.938831 13477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:29.939852 13477 log.cc:824] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:49:29 dist-test-slave-n326 krb5kdc[13057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.189@KRBTEST.COM
I20260811 02:49:29.942747 13335 raft_consensus.cc:348] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } }
I20260811 02:49:29.943005 13335 raft_consensus.cc:374] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:29.943063 13335 raft_consensus.cc:729] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c86594f95ace4affa459281052e4a48c, State: Initialized, Role: FOLLOWER
I20260811 02:49:29.944238 13335 consensus_queue.cc:260] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } }
W20260811 02:49:29.945477 13514 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:49:29.945559 13514 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:49:29.945874 13335 sys_catalog.cc:564] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:49:29.946086 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.920670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42612 (local address 127.21.236.189:39135)
0811 02:49:29.920789 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.920793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.920806 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:49:29.920840 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.920843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.920899 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.920985 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.921781 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.922484 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.924076 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.924421 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.924493 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.941534 (+ 17041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.941539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.941542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.941572 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.943475 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.944066 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.944069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.944072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.944165 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.944519 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.944523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.944525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.944781 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.944994 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.945779 (+ 785us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.945923 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:49:29.944909 13083 master_runner.cc:186] Error getting master registration for 127.21.236.189:39135: OK, instance_id { permanent_uuid: "c86594f95ace4affa459281052e4a48c" instance_seqno: 1786416569656357 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 02:49:29.948410 13083 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:49:29.948868 13521 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } } }
I20260811 02:49:29.949831 13521 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:49:29.950950 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:49:29.951575 13477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e: No bootstrap required, opened a new log
W20260811 02:49:29.951853 13538 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:49:29.951939 13538 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:49:29.956305 13477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } }
I20260811 02:49:29.956565 13477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:29.956621 13477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2787bbffd1114208bed6d32125a02e8e, State: Initialized, Role: FOLLOWER
I20260811 02:49:29.957127 13477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } }
I20260811 02:49:29.954020 13210 master_runner.cc:186] Error getting master registration for 127.21.236.188:32827: OK, instance_id { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" instance_seqno: 1786416569881640 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 02:49:29.957481 13210 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:49:29.958362 13477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [sys.catalog]: configured and running, proceeding with master startup.
W20260811 02:49:29.964303 13570 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:49:29.964390 13570 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:49:29.966898 13347 master_runner.cc:263] No leader master found from master 2787bbffd1114208bed6d32125a02e8e
I20260811 02:49:29.966995 13347 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:49:29.971278 13550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } } }
I20260811 02:49:29.971441 13550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:49:29.972748 13486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.954635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34194 (local address 127.21.236.188:32827)
0811 02:49:29.954748 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.954753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.958822 (+ 4069us) server_negotiation.cc:407] Connection header received
0811 02:49:29.960218 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.960222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.960275 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.960333 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.961420 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.962223 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.963387 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.963759 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.963843 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.968770 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.968775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.968778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.968808 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.970572 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.971170 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.971174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.971176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.971273 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.971619 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.971622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.971624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.971860 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.972021 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.972433 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.972573 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 02:49:29.973987 13207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.953586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43182 (local address 127.21.236.190:42981)
0811 02:49:29.953710 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.953715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.957278 (+ 3563us) server_negotiation.cc:407] Connection header received
0811 02:49:29.957476 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.957480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.957535 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.957592 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.958635 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.959491 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.961691 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.961994 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.962072 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.966309 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.966314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.966317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.966349 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.968399 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.971624 (+ 3225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.971628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.971630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.971726 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.972280 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.972284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.972286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.972520 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.972635 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.973700 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.973820 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 02:49:29.975732 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:29.954023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42628 (local address 127.21.236.189:39135)
0811 02:49:29.954138 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:29.954142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:29.955623 (+ 1481us) server_negotiation.cc:407] Connection header received
0811 02:49:29.955835 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:29.955839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:29.955907 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:29.955966 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:29.958919 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.959775 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.962781 (+ 3006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:29.963137 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:29.963216 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:29.968985 (+ 5769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:29.968989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:29.968992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:29.969018 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:29.971171 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.973100 (+ 1929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.973104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.973106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.973198 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:29.974679 (+ 1481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:29.974682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:29.974684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:29.974908 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:29.975042 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:29.975438 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 02:49:29.975576 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 02:49:30.094691 13494 raft_consensus.cc:482] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:49:30.094938 13494 raft_consensus.cc:504] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } }
I20260811 02:49:30.096190 13494 leader_election.cc:288] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c86594f95ace4affa459281052e4a48c (127.21.236.189:39135), 2787bbffd1114208bed6d32125a02e8e (127.21.236.188:32827)
I20260811 02:49:30.097167 13418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dad952994c3f47b4b79de063e986dd96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2787bbffd1114208bed6d32125a02e8e" is_pre_election: true
I20260811 02:49:30.097527 13418 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad952994c3f47b4b79de063e986dd96 in term 0.
I20260811 02:49:30.098300 13284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dad952994c3f47b4b79de063e986dd96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c86594f95ace4affa459281052e4a48c" is_pre_election: true
I20260811 02:49:30.098587 13284 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad952994c3f47b4b79de063e986dd96 in term 0.
I20260811 02:49:30.099154 13106 leader_election.cc:302] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c86594f95ace4affa459281052e4a48c, dad952994c3f47b4b79de063e986dd96; no voters:
I20260811 02:49:30.100492 13494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:49:30.100591 13494 raft_consensus.cc:482] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:49:30.100637 13494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:30.101956 13494 raft_consensus.cc:504] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } }
I20260811 02:49:30.102264 13494 leader_election.cc:288] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [CANDIDATE]: Term 1 election: Requested vote from peers c86594f95ace4affa459281052e4a48c (127.21.236.189:39135), 2787bbffd1114208bed6d32125a02e8e (127.21.236.188:32827)
I20260811 02:49:30.102768 13418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dad952994c3f47b4b79de063e986dd96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2787bbffd1114208bed6d32125a02e8e"
I20260811 02:49:30.102887 13418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:30.104144 13418 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad952994c3f47b4b79de063e986dd96 in term 1.
I20260811 02:49:30.104650 13284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dad952994c3f47b4b79de063e986dd96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c86594f95ace4affa459281052e4a48c"
I20260811 02:49:30.104794 13284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:30.105960 13284 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad952994c3f47b4b79de063e986dd96 in term 1.
I20260811 02:49:30.106333 13106 leader_election.cc:302] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c86594f95ace4affa459281052e4a48c, dad952994c3f47b4b79de063e986dd96; no voters:
I20260811 02:49:30.106621 13494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:49:30.106897 13494 raft_consensus.cc:686] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [term 1 LEADER]: Becoming Leader. State: Replica: dad952994c3f47b4b79de063e986dd96, State: Running, Role: LEADER
I20260811 02:49:30.107304 13494 consensus_queue.cc:237] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } }
I20260811 02:49:30.109186 13494 sys_catalog.cc:455] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dad952994c3f47b4b79de063e986dd96. Latest consensus state: current_term: 1 leader_uuid: "dad952994c3f47b4b79de063e986dd96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } } }
I20260811 02:49:30.109589 13494 sys_catalog.cc:458] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:30.111409 13581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:49:30.112403 13581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:49:30.118618 13418 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [term 1 FOLLOWER]: Refusing update from remote peer dad952994c3f47b4b79de063e986dd96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:49:30.119511 13578 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:49:30.124050 13550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dad952994c3f47b4b79de063e986dd96. Latest consensus state: current_term: 1 leader_uuid: "dad952994c3f47b4b79de063e986dd96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } } }
I20260811 02:49:30.124178 13550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:49:30.125268 13284 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [term 1 FOLLOWER]: Refusing update from remote peer dad952994c3f47b4b79de063e986dd96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 02:49:30.127804 13381 tablet.cc:2369] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:49:30.128707 13578 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:49:30.132812 13521 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [sys.catalog]: SysCatalogTable state changed. Reason: New leader dad952994c3f47b4b79de063e986dd96. Latest consensus state: current_term: 1 leader_uuid: "dad952994c3f47b4b79de063e986dd96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } } }
I20260811 02:49:30.132959 13521 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:49:30.135269 13578 sys_catalog.cc:455] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dad952994c3f47b4b79de063e986dd96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } } }
I20260811 02:49:30.135396 13578 sys_catalog.cc:458] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:30.136689 13550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dad952994c3f47b4b79de063e986dd96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } } }
I20260811 02:49:30.136806 13550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:49:30.137322 13582 mvcc.cc:205] Tried to move back new op lower bound from 7317162271196811264 to 7317162271163129856. Current Snapshot: MvccSnapshot[applied={T|T < 7317162271196811264}]
I20260811 02:49:30.138942 13494 sys_catalog.cc:455] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dad952994c3f47b4b79de063e986dd96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } } }
I20260811 02:49:30.139092 13494 sys_catalog.cc:458] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:30.140005 13521 sys_catalog.cc:455] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dad952994c3f47b4b79de063e986dd96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad952994c3f47b4b79de063e986dd96" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42981 } } peers { permanent_uuid: "c86594f95ace4affa459281052e4a48c" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 39135 } } peers { permanent_uuid: "2787bbffd1114208bed6d32125a02e8e" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 32827 } } }
I20260811 02:49:30.140126 13521 sys_catalog.cc:458] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:49:30.140234 13587 mvcc.cc:205] Tried to move back new op lower bound from 7317162271196811264 to 7317162271163129856. Current Snapshot: MvccSnapshot[applied={T|T < 7317162271196811264}]
I20260811 02:49:30.141021 13583 mvcc.cc:205] Tried to move back new op lower bound from 7317162271196811264 to 7317162271163129856. Current Snapshot: MvccSnapshot[applied={T|T < 7317162271196811264}]
I20260811 02:49:30.142814 13581 catalog_manager.cc:1347] Generated new cluster ID: 8f8107e2275c4197b4120db3b57273c9
I20260811 02:49:30.142875 13581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:49:30.176954 13581 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:49:30.178007 13581 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:49:30.196130 13581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dad952994c3f47b4b79de063e986dd96: Generated new TSK 0
I20260811 02:49:30.197216 13581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:49:30.953356 13538 catalog_manager.cc:1259] Loaded cluster ID: 8f8107e2275c4197b4120db3b57273c9
I20260811 02:49:30.953449 13538 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c: loading cluster ID for follower catalog manager: success
I20260811 02:49:30.955106 13538 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c: acquiring CA information for follower catalog manager: success
I20260811 02:49:30.956248 13538 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c86594f95ace4affa459281052e4a48c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:49:30.965759 13570 catalog_manager.cc:1259] Loaded cluster ID: 8f8107e2275c4197b4120db3b57273c9
I20260811 02:49:30.965828 13570 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e: loading cluster ID for follower catalog manager: success
I20260811 02:49:30.967450 13570 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e: acquiring CA information for follower catalog manager: success
I20260811 02:49:30.968595 13570 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2787bbffd1114208bed6d32125a02e8e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:49:31.269637 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 13083
I20260811 02:49:31.282210 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 13210
I20260811 02:49:31.294132 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 13347
2026-08-11T02:49:31Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3483 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:49:31 dist-test-slave-n326 krb5kdc[13601](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:49:31 dist-test-slave-n326 krb5kdc[13601](info): set up 2 sockets
Aug 11 02:49:31 dist-test-slave-n326 krb5kdc[13601](info): commencing operation
krb5kdc: starting...
W20260811 02:49:32.628144 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:49:32 dist-test-slave-n326 krb5kdc[13601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:49:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:49:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:49:32.771507 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:43643
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:43643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:32.884455 13625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:32.884785 13625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:32.884858 13625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:32.888983 13625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:49:32.889072 13625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:32.889111 13625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:32.889170 13625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:49:32.889211 13625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:49:32.894623 13625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:43643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:43643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:32.895813 13625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:32.896867 13625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:49:32.905763 13625 server_base.cc:1034] running on GCE node
W20260811 02:49:32.905690 13632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:32.905690 13634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:32.905726 13630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:32.906543 13625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:32.907795 13625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:32.908986 13625 hybrid_clock.cc:648] HybridClock initialized: now 1786416572908970 us; error 37 us; skew 500 ppm
Aug 11 02:49:32 dist-test-slave-n326 krb5kdc[13601](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416572, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:32.915867 13625 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:49:32.916826 13625 webserver.cc:460] Webserver started at http://127.21.236.190:40681/ using document root <none> and password file <none>
I20260811 02:49:32.917363 13625 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:32.917421 13625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:32.917601 13625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:49:32.919737 13625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "461199c5c3924ab684ccf8aaaa84f3da"
format_stamp: "Formatted at 2026-08-11 02:49:32 on dist-test-slave-n326"
server_key: "b756dac1430d1b8bd3525c09f009f82f"
server_key_iv: "70eac2db9dab66548d7315d53b615037"
server_key_version: "encryptionkey@0"
I20260811 02:49:32.920259 13625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "461199c5c3924ab684ccf8aaaa84f3da"
format_stamp: "Formatted at 2026-08-11 02:49:32 on dist-test-slave-n326"
server_key: "b756dac1430d1b8bd3525c09f009f82f"
server_key_iv: "70eac2db9dab66548d7315d53b615037"
server_key_version: "encryptionkey@0"
I20260811 02:49:32.924150 13625 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:49:32.926527 13648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:32.927834 13625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:49:32.927987 13625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "461199c5c3924ab684ccf8aaaa84f3da"
format_stamp: "Formatted at 2026-08-11 02:49:32 on dist-test-slave-n326"
server_key: "b756dac1430d1b8bd3525c09f009f82f"
server_key_iv: "70eac2db9dab66548d7315d53b615037"
server_key_version: "encryptionkey@0"
I20260811 02:49:32.928112 13625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:32.941169 13625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:32.944511 13625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:32.944733 13625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:32.952497 13625 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:43643
I20260811 02:49:32.952569 13733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:43643 every 8 connection(s)
I20260811 02:49:32.953512 13625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:49:32.956290 13734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:32.959388 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 13625
I20260811 02:49:32.959517 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:49:32.959837 22450 external_mini_cluster.cc:1428] Setting key 9d7cf0eb692731a1f9787623da23d205
I20260811 02:49:32.962417 13734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da: Bootstrap starting.
I20260811 02:49:32.964973 13734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:32.965886 13734 log.cc:824] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:32.968240 13734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da: No bootstrap required, opened a new log
Aug 11 02:49:32 dist-test-slave-n326 krb5kdc[13601](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:32.972390 13734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "461199c5c3924ab684ccf8aaaa84f3da" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43643 } }
I20260811 02:49:32.972671 13734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:32.972775 13734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 461199c5c3924ab684ccf8aaaa84f3da, State: Initialized, Role: FOLLOWER
I20260811 02:49:32.973278 13734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "461199c5c3924ab684ccf8aaaa84f3da" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43643 } }
I20260811 02:49:32.973445 13734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:49:32.973513 13734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:49:32.973639 13734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:32.974797 13734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "461199c5c3924ab684ccf8aaaa84f3da" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43643 } }
I20260811 02:49:32.975178 13734 leader_election.cc:302] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 461199c5c3924ab684ccf8aaaa84f3da; no voters:
I20260811 02:49:32.975507 13734 leader_election.cc:288] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:49:32.975657 13740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:49:32.975900 13740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [term 1 LEADER]: Becoming Leader. State: Replica: 461199c5c3924ab684ccf8aaaa84f3da, State: Running, Role: LEADER
I20260811 02:49:32.975988 13737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:32.961265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38876 (local address 127.21.236.190:43643)
0811 02:49:32.961742 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:32.961758 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:32.961794 (+ 36us) server_negotiation.cc:407] Connection header received
0811 02:49:32.962412 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:32.962426 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:32.962814 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:32.963172 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:32.964119 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:32.965225 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:32.966414 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:32.967042 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:32.967233 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:32.969955 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:32.969963 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:32.969980 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:32.970011 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:32.972607 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:32.973149 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:32.973155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:32.973160 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:32.973295 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:32.973684 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:32.973690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:32.973693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:32.974194 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:32.974391 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:32.974623 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 02:49:32.974930 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":151,"threads_started":1}
I20260811 02:49:32.976255 13740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "461199c5c3924ab684ccf8aaaa84f3da" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43643 } }
I20260811 02:49:32.976755 13734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:49:32.977443 13742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 461199c5c3924ab684ccf8aaaa84f3da. Latest consensus state: current_term: 1 leader_uuid: "461199c5c3924ab684ccf8aaaa84f3da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "461199c5c3924ab684ccf8aaaa84f3da" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43643 } } }
I20260811 02:49:32.977869 13743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "461199c5c3924ab684ccf8aaaa84f3da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "461199c5c3924ab684ccf8aaaa84f3da" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43643 } } }
I20260811 02:49:32.978047 13743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:32.978047 13742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:32.978607 13755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:49:32.979560 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:49:32.981921 13755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:49:32.987352 13755 catalog_manager.cc:1347] Generated new cluster ID: 80d789c535ca45c7a0e3ec4a286d8ff2
I20260811 02:49:32.987427 13755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:49:33.008209 13755 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:49:33.009543 13755 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:49:33.019981 13755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 461199c5c3924ab684ccf8aaaa84f3da: Generated new TSK 0
I20260811 02:49:33.020704 13755 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:49:33.035522 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 13625
2026-08-11T02:49:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1746 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:49:33 dist-test-slave-n326 krb5kdc[13772](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:49:33 dist-test-slave-n326 krb5kdc[13772](info): set up 2 sockets
Aug 11 02:49:33 dist-test-slave-n326 krb5kdc[13772](info): commencing operation
krb5kdc: starting...
W20260811 02:49:34.410046 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.344s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:49:34 dist-test-slave-n326 krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:49:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:49:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:49:34.546689 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:43417
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:43417
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:34.658746 13798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:34.659068 13798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:34.659190 13798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:34.663175 13798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:49:34.663269 13798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:34.663311 13798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:34.663348 13798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:49:34.663383 13798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:49:34.668610 13798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:43417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:43417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:34.669714 13798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:34.670742 13798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:34.679383 13805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:34.679428 13804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:34.679402 13798 server_base.cc:1034] running on GCE node
W20260811 02:49:34.679682 13811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:34.680291 13798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:34.681588 13798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:34.682807 13798 hybrid_clock.cc:648] HybridClock initialized: now 1786416574682786 us; error 32 us; skew 500 ppm
Aug 11 02:49:34 dist-test-slave-n326 krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416574, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:34.685889 13798 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:49:34.686839 13798 webserver.cc:460] Webserver started at http://127.21.236.190:41761/ using document root <none> and password file <none>
I20260811 02:49:34.687500 13798 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:34.687570 13798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:34.687752 13798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:49:34.689945 13798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "9b613f24f6cb47f085dcce9afd470cd5"
format_stamp: "Formatted at 2026-08-11 02:49:34 on dist-test-slave-n326"
server_key: "d2f94fdc789d82b279c4957840f2413d"
server_key_iv: "eba7e19009e3826b50dd6b9ed00a59ce"
server_key_version: "encryptionkey@0"
I20260811 02:49:34.690524 13798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "9b613f24f6cb47f085dcce9afd470cd5"
format_stamp: "Formatted at 2026-08-11 02:49:34 on dist-test-slave-n326"
server_key: "d2f94fdc789d82b279c4957840f2413d"
server_key_iv: "eba7e19009e3826b50dd6b9ed00a59ce"
server_key_version: "encryptionkey@0"
I20260811 02:49:34.694515 13798 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:49:34.697177 13823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:34.698472 13798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:49:34.698616 13798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "9b613f24f6cb47f085dcce9afd470cd5"
format_stamp: "Formatted at 2026-08-11 02:49:34 on dist-test-slave-n326"
server_key: "d2f94fdc789d82b279c4957840f2413d"
server_key_iv: "eba7e19009e3826b50dd6b9ed00a59ce"
server_key_version: "encryptionkey@0"
I20260811 02:49:34.698726 13798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:34.727777 13798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:34.731613 13798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:34.731791 13798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:34.739781 13798 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:43417
I20260811 02:49:34.740136 13908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:43417 every 8 connection(s)
I20260811 02:49:34.740844 13798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:49:34.743906 13909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:34.744530 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 13798
I20260811 02:49:34.744637 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:49:34.744987 22450 external_mini_cluster.cc:1428] Setting key f8d365f652b7a89853eebf526ad86b17
I20260811 02:49:34.750805 13909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Bootstrap starting.
I20260811 02:49:34.753839 13909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:34.754930 13909 log.cc:824] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:49:34 dist-test-slave-n326 krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:34.757252 13909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: No bootstrap required, opened a new log
I20260811 02:49:34.762485 13916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:34.746587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47178 (local address 127.21.236.190:43417)
0811 02:49:34.747127 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:34.747153 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:34.747191 (+ 38us) server_negotiation.cc:407] Connection header received
0811 02:49:34.747889 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:34.747908 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:34.748272 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:34.748622 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:34.749661 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:34.750766 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:34.752081 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:34.752668 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:34.752814 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:34.756241 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:34.756247 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:34.756262 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:34.756292 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:34.758663 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:34.759215 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:34.759225 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:34.759232 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:34.759410 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:34.759728 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:34.759732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:34.759735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:34.760401 (+ 666us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:34.760619 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:34.760921 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 02:49:34.761304 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":164,"threads_started":1}
I20260811 02:49:34.762676 13909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } }
I20260811 02:49:34.763031 13909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:34.763124 13909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b613f24f6cb47f085dcce9afd470cd5, State: Initialized, Role: FOLLOWER
I20260811 02:49:34.763649 13909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } }
I20260811 02:49:34.763849 13909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:49:34.763939 13909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:49:34.764048 13909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:34.765398 13909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } }
I20260811 02:49:34.765728 13909 leader_election.cc:302] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b613f24f6cb47f085dcce9afd470cd5; no voters:
I20260811 02:49:34.766021 13909 leader_election.cc:288] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:49:34.766176 13921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:49:34.766425 13921 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 1 LEADER]: Becoming Leader. State: Replica: 9b613f24f6cb47f085dcce9afd470cd5, State: Running, Role: LEADER
I20260811 02:49:34.766798 13921 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } }
I20260811 02:49:34.767455 13909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:49:34.768658 13922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b613f24f6cb47f085dcce9afd470cd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } } }
I20260811 02:49:34.768796 13922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:34.769090 13923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b613f24f6cb47f085dcce9afd470cd5. Latest consensus state: current_term: 1 leader_uuid: "9b613f24f6cb47f085dcce9afd470cd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } } }
I20260811 02:49:34.769186 13923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:34.769236 13934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:49:34.773182 13934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:49:34.779314 13934 catalog_manager.cc:1347] Generated new cluster ID: 876742b78fe54189a0df9d683f8e01f2
I20260811 02:49:34.779397 13934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:49:34.804124 13934 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:49:34.805462 13934 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:49:34.811456 13934 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Generated new TSK 0
I20260811 02:49:34.812206 13934 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:49:34.850622 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43417
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:34.969444 13960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:34.969750 13960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:34.969826 13960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:34.973912 13960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:34.974020 13960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:34.974123 13960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:49:34.979836 13960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:34.980989 13960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:34.982033 13960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:34.992012 13971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:34.992033 13966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:34.992007 13968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:34.992309 13960 server_base.cc:1034] running on GCE node
I20260811 02:49:34.992924 13960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:34.993549 13960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:34.994719 13960 hybrid_clock.cc:648] HybridClock initialized: now 1786416574994703 us; error 28 us; skew 500 ppm
Aug 11 02:49:34 dist-test-slave-n326 krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416574, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:34.997553 13960 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:49:34.998503 13960 webserver.cc:460] Webserver started at http://127.21.236.129:35045/ using document root <none> and password file <none>
I20260811 02:49:34.999140 13960 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:34.999210 13960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:34.999433 13960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:49:35.001560 13960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "d1d236ed396945fea79e9f4c17bdb879"
format_stamp: "Formatted at 2026-08-11 02:49:34 on dist-test-slave-n326"
server_key: "a769efc3966480a236c11162ccf42deb"
server_key_iv: "173c8af639d0f09917a4aea6134cf929"
server_key_version: "encryptionkey@0"
I20260811 02:49:35.002189 13960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "d1d236ed396945fea79e9f4c17bdb879"
format_stamp: "Formatted at 2026-08-11 02:49:34 on dist-test-slave-n326"
server_key: "a769efc3966480a236c11162ccf42deb"
server_key_iv: "173c8af639d0f09917a4aea6134cf929"
server_key_version: "encryptionkey@0"
I20260811 02:49:35.006206 13960 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:49:35.009021 13981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:35.010434 13960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 02:49:35.010610 13960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "d1d236ed396945fea79e9f4c17bdb879"
format_stamp: "Formatted at 2026-08-11 02:49:34 on dist-test-slave-n326"
server_key: "a769efc3966480a236c11162ccf42deb"
server_key_iv: "173c8af639d0f09917a4aea6134cf929"
server_key_version: "encryptionkey@0"
I20260811 02:49:35.010733 13960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:35.027786 13960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:35.030915 13960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:35.031186 13960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:35.031797 13960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:35.032639 13960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:35.032712 13960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:35.032788 13960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:35.032843 13960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:35.042106 13960 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:42999
I20260811 02:49:35.042127 14197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:42999 every 8 connection(s)
I20260811 02:49:35.043017 13960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:49:35.048789 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 13960
I20260811 02:49:35.048892 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:49:35.049185 22450 external_mini_cluster.cc:1428] Setting key 8d43c5e9bc4eaa881ceb3b48e6de07c1
Aug 11 02:49:35 dist-test-slave-n326 krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416574, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:35.058264 13916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.044894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:36303 (local address 127.21.236.190:43417)
0811 02:49:35.045054 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:35.045059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:35.045969 (+ 910us) server_negotiation.cc:407] Connection header received
0811 02:49:35.047364 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:35.047367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:35.047415 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:35.047490 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:35.048616 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.049277 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.050676 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.050901 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.050983 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.054274 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:35.054277 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:35.054279 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:35.054302 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:35.055909 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:35.056535 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:35.056538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:35.056539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:35.056623 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:35.057077 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:35.057081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:35.057082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:35.057255 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:35.057331 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:35.058017 (+ 686us) server_negotiation.cc:299] Negotiation successful
0811 02:49:35.058142 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 02:49:35.059669 14201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.045347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:36303)
0811 02:49:35.045804 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:35.045849 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:35.047159 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:35.047619 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:35.047625 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:35.047971 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:35.048451 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.048464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.049389 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.049394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.050534 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.050544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.051108 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.051117 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.051347 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.051974 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:35.052001 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:35.054097 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:35.056038 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:35.056046 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:35.056065 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:35.056436 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:35.056723 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:35.056728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:35.056731 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:35.056941 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:35.057351 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:35.057360 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:35.057732 (+ 372us) client_negotiation.cc:769] Sending connection context
0811 02:49:35.057973 (+ 241us) client_negotiation.cc:240] Negotiation successful
0811 02:49:35.058496 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
I20260811 02:49:35.061019 14198 heartbeater.cc:347] Connected to a master server at 127.21.236.190:43417
I20260811 02:49:35.061251 14198 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:35.061795 14198 heartbeater.cc:511] Master 127.21.236.190:43417 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
I20260811 02:49:35.063386 13854 ts_manager.cc:194] Registered new tserver with Master: d1d236ed396945fea79e9f4c17bdb879 (127.21.236.129:42999)
I20260811 02:49:35.064994 13854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:36303
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:49:35.080497 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43417
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:35.194806 14207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:35.195139 14207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:35.195191 14207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:35.199380 14207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:35.199446 14207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:35.199505 14207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:49:35.204743 14207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:35.205737 14207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:35.206744 14207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:35.216548 14219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:35.216578 14221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:35.216599 14216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:35.217067 14207 server_base.cc:1034] running on GCE node
I20260811 02:49:35.217471 14207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:35.218130 14207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:35.219304 14207 hybrid_clock.cc:648] HybridClock initialized: now 1786416575219279 us; error 38 us; skew 500 ppm
Aug 11 02:49:35 dist-test-slave-n326 krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:35.222064 14207 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:49:35.223004 14207 webserver.cc:460] Webserver started at http://127.21.236.130:45147/ using document root <none> and password file <none>
I20260811 02:49:35.223630 14207 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:35.223697 14207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:35.223869 14207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:49:35.225885 14207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "1601bebbded246e3874fc777ac130b88"
format_stamp: "Formatted at 2026-08-11 02:49:35 on dist-test-slave-n326"
server_key: "a7ca90cf4d8d51665593e4acc854f1b6"
server_key_iv: "3b100dca3be3624d89409eb5063f9708"
server_key_version: "encryptionkey@0"
I20260811 02:49:35.226414 14207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "1601bebbded246e3874fc777ac130b88"
format_stamp: "Formatted at 2026-08-11 02:49:35 on dist-test-slave-n326"
server_key: "a7ca90cf4d8d51665593e4acc854f1b6"
server_key_iv: "3b100dca3be3624d89409eb5063f9708"
server_key_version: "encryptionkey@0"
I20260811 02:49:35.230393 14207 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:49:35.232843 14236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:35.234077 14207 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:49:35.234308 14207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "1601bebbded246e3874fc777ac130b88"
format_stamp: "Formatted at 2026-08-11 02:49:35 on dist-test-slave-n326"
server_key: "a7ca90cf4d8d51665593e4acc854f1b6"
server_key_iv: "3b100dca3be3624d89409eb5063f9708"
server_key_version: "encryptionkey@0"
I20260811 02:49:35.234421 14207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:35.262234 14207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:35.265331 14207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:35.265552 14207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:35.266240 14207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:35.267184 14207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:35.267236 14207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:35.267303 14207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:35.267342 14207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:35.276892 14207 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:41397
I20260811 02:49:35.276919 14442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:41397 every 8 connection(s)
I20260811 02:49:35.277858 14207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
Aug 11 02:49:35 dist-test-slave-n326 krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:35.288080 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 14207
I20260811 02:49:35.288178 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:49:35.288465 22450 external_mini_cluster.cc:1428] Setting key 8de0bae567a77b4c7fb9ce86e27edb9c
I20260811 02:49:35.292505 13916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.279698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:36215 (local address 127.21.236.190:43417)
0811 02:49:35.279838 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:35.279842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:35.280740 (+ 898us) server_negotiation.cc:407] Connection header received
0811 02:49:35.282144 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:35.282148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:35.282211 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:35.282313 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:35.283535 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.284071 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.285239 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.285470 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.285586 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.288524 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:35.288531 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:35.288534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:35.288585 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:35.290317 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:35.290882 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:35.290885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:35.290892 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:35.290979 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:35.291379 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:35.291384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:35.291385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:35.291570 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:35.291682 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:35.292210 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 02:49:35.292340 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:49:35.293430 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.280109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:36215)
0811 02:49:35.280565 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:35.280628 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:35.281912 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:35.282490 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:35.282497 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:35.282876 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:35.283383 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.283397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.284216 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.284219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.285107 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.285117 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.285639 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.285644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.285825 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.286442 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:35.286459 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:35.288336 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:35.290437 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:35.290444 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:35.290460 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:35.290768 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:35.291117 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:35.291120 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:35.291122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:35.291275 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:35.291682 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:35.291690 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:35.291957 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 02:49:35.292148 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 02:49:35.292500 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":121,"threads_started":1}
I20260811 02:49:35.294739 14444 heartbeater.cc:347] Connected to a master server at 127.21.236.190:43417
I20260811 02:49:35.295009 14444 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:35.295557 14444 heartbeater.cc:511] Master 127.21.236.190:43417 requested a full tablet report, sending...
I20260811 02:49:35.296746 13854 ts_manager.cc:194] Registered new tserver with Master: 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397)
I20260811 02:49:35.297760 13854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:36215
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:49:35.322762 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43417
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:35.459275 14449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:35.459652 14449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:35.459726 14449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:35.468245 14449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:35.468356 14449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:35.468485 14449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:49:35.479353 14449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:43417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:35.480443 14449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:35.481645 14449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:35.491432 14468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:35.491487 14460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:35.491457 14463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:35.491757 14449 server_base.cc:1034] running on GCE node
I20260811 02:49:35.492161 14449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:35.492765 14449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:35.493950 14449 hybrid_clock.cc:648] HybridClock initialized: now 1786416575493919 us; error 40 us; skew 500 ppm
Aug 11 02:49:35 dist-test-slave-n326 krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:35.497174 14449 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:49:35.498167 14449 webserver.cc:460] Webserver started at http://127.21.236.131:45923/ using document root <none> and password file <none>
I20260811 02:49:35.498785 14449 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:35.498889 14449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:35.499234 14449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:49:35.501535 14449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "4be7e38240de4b3b9c2b61aa129f4a6f"
format_stamp: "Formatted at 2026-08-11 02:49:35 on dist-test-slave-n326"
server_key: "2613966eb5a013778c94a6d4bef3ed60"
server_key_iv: "5fc8166c9d157303447b30782a1f3bfb"
server_key_version: "encryptionkey@0"
I20260811 02:49:35.502153 14449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "4be7e38240de4b3b9c2b61aa129f4a6f"
format_stamp: "Formatted at 2026-08-11 02:49:35 on dist-test-slave-n326"
server_key: "2613966eb5a013778c94a6d4bef3ed60"
server_key_iv: "5fc8166c9d157303447b30782a1f3bfb"
server_key_version: "encryptionkey@0"
I20260811 02:49:35.506168 14449 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:49:35.508917 14480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:35.510167 14449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 02:49:35.510317 14449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "4be7e38240de4b3b9c2b61aa129f4a6f"
format_stamp: "Formatted at 2026-08-11 02:49:35 on dist-test-slave-n326"
server_key: "2613966eb5a013778c94a6d4bef3ed60"
server_key_iv: "5fc8166c9d157303447b30782a1f3bfb"
server_key_version: "encryptionkey@0"
I20260811 02:49:35.510474 14449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:35.552351 14449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:35.556283 14449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:35.556540 14449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:35.557232 14449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:35.558192 14449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:35.558279 14449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:35.558358 14449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:35.558426 14449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:35.568740 14449 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:43269
I20260811 02:49:35.568919 14679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:43269 every 8 connection(s)
I20260811 02:49:35.569931 14449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:49:35.572337 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 14449
I20260811 02:49:35.572479 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:49:35.572806 22450 external_mini_cluster.cc:1428] Setting key 0c39bc449f8a395da6be8cfe94d9c74a
Aug 11 02:49:35 dist-test-slave-n326 krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:35.585798 13916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.572123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:53671 (local address 127.21.236.190:43417)
0811 02:49:35.572273 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:35.572278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:35.573425 (+ 1147us) server_negotiation.cc:407] Connection header received
0811 02:49:35.575450 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:35.575453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:35.575501 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:35.575574 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:35.576795 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.577379 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.578612 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.578857 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.578953 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.581696 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:35.581700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:35.581702 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:35.581729 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:35.583556 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:35.584114 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:35.584116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:35.584118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:35.584201 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:35.584567 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:35.584570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:35.584572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:35.584785 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:35.584888 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:35.585482 (+ 594us) server_negotiation.cc:299] Negotiation successful
0811 02:49:35.585621 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:49:35.586771 14685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.572673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:53671)
0811 02:49:35.573248 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:35.573313 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:35.575220 (+ 1907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:35.575736 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:35.575744 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:35.576159 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:35.576665 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.576679 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.577519 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.577523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.578490 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.578500 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.579021 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.579024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.579262 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.579884 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:35.579902 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:35.581505 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:35.583677 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:35.583682 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:35.583695 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:35.584014 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:35.584289 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:35.584291 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:35.584292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:35.584437 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:35.584922 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:35.584928 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:35.585199 (+ 271us) client_negotiation.cc:769] Sending connection context
0811 02:49:35.585370 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 02:49:35.585642 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":156,"threads_started":1}
I20260811 02:49:35.588009 14680 heartbeater.cc:347] Connected to a master server at 127.21.236.190:43417
I20260811 02:49:35.588238 14680 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:35.588692 14680 heartbeater.cc:511] Master 127.21.236.190:43417 requested a full tablet report, sending...
I20260811 02:49:35.589794 13854 ts_manager.cc:194] Registered new tserver with Master: 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269)
I20260811 02:49:35.590468 13854 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:53671
I20260811 02:49:35.599345 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:49:35.609239 13916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.601525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47188 (local address 127.21.236.190:43417)
0811 02:49:35.601692 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:35.601697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:35.601933 (+ 236us) server_negotiation.cc:407] Connection header received
0811 02:49:35.602025 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:35.602027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:35.602071 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:35.602140 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:35.602888 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.603595 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.604448 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.604680 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.604770 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.606087 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:35.606090 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:35.606092 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:35.606117 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:35.607670 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:35.608057 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:35.608060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:35.608061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:35.608145 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:35.608392 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:35.608395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:35.608397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:35.608612 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:35.608719 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:35.608915 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 02:49:35.609009 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:49:35.612186 13854 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:49:35.615648 13854 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:49:35.630195 14715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.624444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:41397 (local address 127.0.0.1:35690)
0811 02:49:35.625450 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:35.625466 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:35.625559 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:35.626128 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:35.626130 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:35.626143 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:35.626226 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.626235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.627342 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.627348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.628722 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.628734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.629627 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.629631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.629748 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.629787 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 02:49:35.629894 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 02:49:35.630036 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"thread_start_us":111,"threads_started":1}
I20260811 02:49:35.630537 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.624816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35690 (local address 127.21.236.130:41397)
0811 02:49:35.625272 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:35.625285 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:35.625675 (+ 390us) server_negotiation.cc:407] Connection header received
0811 02:49:35.625784 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:35.625792 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:35.625973 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:35.626134 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:35.626316 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.627213 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.628862 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.629496 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.629637 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.629776 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:35.630275 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 02:49:35.630389 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":156,"threads_started":1}
I20260811 02:49:35.630525 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.624322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43640 (local address 127.21.236.129:42999)
0811 02:49:35.625361 (+ 1039us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:35.625368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:35.625394 (+ 26us) server_negotiation.cc:407] Connection header received
0811 02:49:35.625493 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:35.625499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:35.625721 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:35.625867 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:35.626341 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.627526 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.628906 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.629728 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.629854 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.629993 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:35.630057 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 02:49:35.630170 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935,"thread_start_us":88,"threads_started":1}
I20260811 02:49:35.630697 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.624390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56018 (local address 127.21.236.131:43269)
0811 02:49:35.624831 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:35.624841 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:35.625629 (+ 788us) server_negotiation.cc:407] Connection header received
0811 02:49:35.625846 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:35.625851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:35.626001 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:35.626143 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:35.626577 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.627721 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.629039 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.629827 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.629933 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.630035 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:35.630413 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 02:49:35.630524 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":109,"threads_started":1}
I20260811 02:49:35.630195 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.624153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42999 (local address 127.0.0.1:43640)
0811 02:49:35.625009 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:35.625059 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:35.625202 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:35.626007 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:35.626012 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:35.626031 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:35.626184 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.626193 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.627651 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.627654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.628809 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.628821 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.629858 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.629864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.629951 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.629970 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 02:49:35.630028 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 02:49:35.630076 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":46,"thread_start_us":88,"threads_started":1}
I20260811 02:49:35.631125 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.624153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:43269 (local address 127.0.0.1:56018)
0811 02:49:35.625532 (+ 1379us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:35.625545 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:35.625699 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:35.626312 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:35.626315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:35.626332 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:35.626434 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.626440 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.627887 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.627890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.628900 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.628924 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.630101 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.630105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.630200 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.630223 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:49:35.630296 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 02:49:35.630977 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1324,"mutex_wait_us":846,"thread_start_us":94,"threads_started":1}
I20260811 02:49:35.633437 14325 tablet_service.cc:1467] Processing CreateTablet for tablet 2861ae426df84b59b231a55c0b4a3cd2 (DEFAULT_TABLE table=test-table [id=5a641e02304449fcb2e0a0240fac2d2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:49:35.633872 14557 tablet_service.cc:1467] Processing CreateTablet for tablet 2861ae426df84b59b231a55c0b4a3cd2 (DEFAULT_TABLE table=test-table [id=5a641e02304449fcb2e0a0240fac2d2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:49:35.633872 14077 tablet_service.cc:1467] Processing CreateTablet for tablet 2861ae426df84b59b231a55c0b4a3cd2 (DEFAULT_TABLE table=test-table [id=5a641e02304449fcb2e0a0240fac2d2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:49:35.634496 14325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2861ae426df84b59b231a55c0b4a3cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:35.634899 14077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2861ae426df84b59b231a55c0b4a3cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:35.634899 14557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2861ae426df84b59b231a55c0b4a3cd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:35.641139 14718 tablet_bootstrap.cc:492] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88: Bootstrap starting.
I20260811 02:49:35.642526 14719 tablet_bootstrap.cc:492] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: Bootstrap starting.
I20260811 02:49:35.643412 14720 tablet_bootstrap.cc:492] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f: Bootstrap starting.
I20260811 02:49:35.644061 14718 tablet_bootstrap.cc:654] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:35.644788 14719 tablet_bootstrap.cc:654] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:35.645040 14718 log.cc:824] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:35.645293 14720 tablet_bootstrap.cc:654] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:35.645736 14719 log.cc:824] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:35.646621 14720 log.cc:824] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:35.647323 14718 tablet_bootstrap.cc:492] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88: No bootstrap required, opened a new log
I20260811 02:49:35.647526 14718 ts_tablet_manager.cc:1397] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260811 02:49:35.647869 14719 tablet_bootstrap.cc:492] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: No bootstrap required, opened a new log
I20260811 02:49:35.648044 14719 ts_tablet_manager.cc:1397] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:49:35.648475 14720 tablet_bootstrap.cc:492] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f: No bootstrap required, opened a new log
I20260811 02:49:35.648671 14720 ts_tablet_manager.cc:1397] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:49:35.651402 14720 raft_consensus.cc:348] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } }
I20260811 02:49:35.651641 14720 raft_consensus.cc:374] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:35.651695 14720 raft_consensus.cc:729] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4be7e38240de4b3b9c2b61aa129f4a6f, State: Initialized, Role: FOLLOWER
I20260811 02:49:35.651530 14718 raft_consensus.cc:348] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } }
I20260811 02:49:35.651793 14718 raft_consensus.cc:374] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:35.651855 14718 raft_consensus.cc:729] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1601bebbded246e3874fc777ac130b88, State: Initialized, Role: FOLLOWER
I20260811 02:49:35.652226 14720 consensus_queue.cc:260] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } }
I20260811 02:49:35.652311 14718 consensus_queue.cc:260] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } }
I20260811 02:49:35.652303 14719 raft_consensus.cc:348] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } }
I20260811 02:49:35.652522 14719 raft_consensus.cc:374] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:35.652593 14719 raft_consensus.cc:729] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d236ed396945fea79e9f4c17bdb879, State: Initialized, Role: FOLLOWER
I20260811 02:49:35.652961 14719 consensus_queue.cc:260] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } }
I20260811 02:49:35.653257 14720 ts_tablet_manager.cc:1428] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:49:35.653301 14718 ts_tablet_manager.cc:1428] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 02:49:35.653409 14680 heartbeater.cc:503] Master 127.21.236.190:43417 was elected leader, sending a full tablet report...
I20260811 02:49:35.653621 14444 heartbeater.cc:503] Master 127.21.236.190:43417 was elected leader, sending a full tablet report...
I20260811 02:49:35.653812 14719 ts_tablet_manager.cc:1428] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:49:35.654000 14198 heartbeater.cc:503] Master 127.21.236.190:43417 was elected leader, sending a full tablet report...
I20260811 02:49:35.657486 14728 raft_consensus.cc:482] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:49:35.657949 14728 raft_consensus.cc:504] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } }
I20260811 02:49:35.658960 14728 leader_election.cc:288] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1d236ed396945fea79e9f4c17bdb879 (127.21.236.129:42999), 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269)
I20260811 02:49:35.663071 14446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.659180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42999 (local address 127.21.236.130:35663)
0811 02:49:35.659317 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:35.659334 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:35.659418 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:35.659773 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:35.659776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:35.659790 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:35.659941 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.659948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.660999 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.661002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.661835 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.661845 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.662605 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.662609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.662765 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.662787 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:49:35.662857 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 02:49:35.662938 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":45}
I20260811 02:49:35.663146 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.659433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:35663 (local address 127.21.236.129:42999)
0811 02:49:35.659537 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:35.659541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:35.659566 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:49:35.659601 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:35.659603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:35.659664 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:35.659742 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:35.660055 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.660862 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.661964 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.662478 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.662572 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.662632 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:35.662909 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 02:49:35.662989 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 02:49:35.663192 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.659281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:45643 (local address 127.21.236.131:43269)
0811 02:49:35.659408 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:35.659413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:35.659659 (+ 246us) server_negotiation.cc:407] Connection header received
0811 02:49:35.659735 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:35.659738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:35.659778 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:35.659847 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:35.660124 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.660862 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.662153 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.662562 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.662676 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.662735 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:35.662958 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 02:49:35.663031 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:49:35.663213 14732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.659220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:43269 (local address 127.21.236.130:45643)
0811 02:49:35.659568 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:35.659579 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:35.659663 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:35.659859 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:35.659862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:35.659873 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:35.660007 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.660016 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.660980 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.660984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.662025 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:35.662037 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.662693 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.662696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:35.662812 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.662830 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 02:49:35.662929 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 02:49:35.662993 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":60,"thread_start_us":92,"threads_started":1}
I20260811 02:49:35.663741 14115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2861ae426df84b59b231a55c0b4a3cd2" candidate_uuid: "1601bebbded246e3874fc777ac130b88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1d236ed396945fea79e9f4c17bdb879" is_pre_election: true
I20260811 02:49:35.663815 14600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2861ae426df84b59b231a55c0b4a3cd2" candidate_uuid: "1601bebbded246e3874fc777ac130b88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" is_pre_election: true
I20260811 02:49:35.663995 14115 raft_consensus.cc:2454] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1601bebbded246e3874fc777ac130b88 in term 0.
I20260811 02:49:35.664155 14600 raft_consensus.cc:2454] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1601bebbded246e3874fc777ac130b88 in term 0.
I20260811 02:49:35.664489 14244 leader_election.cc:302] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1601bebbded246e3874fc777ac130b88, d1d236ed396945fea79e9f4c17bdb879; no voters:
I20260811 02:49:35.664750 14728 raft_consensus.cc:2781] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:49:35.664826 14728 raft_consensus.cc:482] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:49:35.664860 14728 raft_consensus.cc:3037] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:35.665992 14728 raft_consensus.cc:504] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } }
I20260811 02:49:35.666342 14728 leader_election.cc:288] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [CANDIDATE]: Term 1 election: Requested vote from peers d1d236ed396945fea79e9f4c17bdb879 (127.21.236.129:42999), 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269)
I20260811 02:49:35.666718 14600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2861ae426df84b59b231a55c0b4a3cd2" candidate_uuid: "1601bebbded246e3874fc777ac130b88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f"
I20260811 02:49:35.666839 14600 raft_consensus.cc:3037] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:35.666887 14115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2861ae426df84b59b231a55c0b4a3cd2" candidate_uuid: "1601bebbded246e3874fc777ac130b88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1d236ed396945fea79e9f4c17bdb879"
I20260811 02:49:35.667016 14115 raft_consensus.cc:3037] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:35.667958 14600 raft_consensus.cc:2454] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1601bebbded246e3874fc777ac130b88 in term 1.
I20260811 02:49:35.668135 14115 raft_consensus.cc:2454] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1601bebbded246e3874fc777ac130b88 in term 1.
I20260811 02:49:35.668257 14240 leader_election.cc:302] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1601bebbded246e3874fc777ac130b88, 4be7e38240de4b3b9c2b61aa129f4a6f; no voters:
I20260811 02:49:35.668437 14728 raft_consensus.cc:2781] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:49:35.668682 14728 raft_consensus.cc:686] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [term 1 LEADER]: Becoming Leader. State: Replica: 1601bebbded246e3874fc777ac130b88, State: Running, Role: LEADER
I20260811 02:49:35.669006 14728 consensus_queue.cc:237] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } }
I20260811 02:49:35.672231 13854 catalog_manager.cc:5495] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1601bebbded246e3874fc777ac130b88 (127.21.236.130). New cstate: current_term: 1 leader_uuid: "1601bebbded246e3874fc777ac130b88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } health_report { overall_health: HEALTHY } } }
I20260811 02:49:35.691744 14716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.688454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.21.236.130:41397)
0811 02:49:35.688580 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:35.688584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:35.688600 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:49:35.688673 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:35.688675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:35.688729 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:35.688825 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:49:35.689052 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.689549 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.690367 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:35.690581 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:35.690665 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:35.690936 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:49:35.691385 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:49:35.691477 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:35.691561 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 02:49:35.691616 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:49:35.697466 14600 raft_consensus.cc:1261] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 1 FOLLOWER]: Refusing update from remote peer 1601bebbded246e3874fc777ac130b88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:49:35.697466 14115 raft_consensus.cc:1261] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Refusing update from remote peer 1601bebbded246e3874fc777ac130b88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:49:35.698243 14733 consensus_queue.cc:1035] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:49:35.698463 14728 consensus_queue.cc:1035] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:49:35.706164 14740 mvcc.cc:205] Tried to move back new op lower bound from 7317162294049632256 to 7317162293942550528. Current Snapshot: MvccSnapshot[applied={T|T < 7317162294049632256}]
I20260811 02:49:35.706817 14739 mvcc.cc:205] Tried to move back new op lower bound from 7317162294049632256 to 7317162293942550528. Current Snapshot: MvccSnapshot[applied={T|T < 7317162294049632256}]
I20260811 02:49:35.716908 13848 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:49:35.717075 13848 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:49:35.719692 13848 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Sending DeleteTablet for 3 replicas of tablet 2861ae426df84b59b231a55c0b4a3cd2
I20260811 02:49:35.720522 14077 tablet_service.cc:1514] Processing DeleteTablet for tablet 2861ae426df84b59b231a55c0b4a3cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:49:35 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:43640
I20260811 02:49:35.720512 14557 tablet_service.cc:1514] Processing DeleteTablet for tablet 2861ae426df84b59b231a55c0b4a3cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:49:35 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:56018
I20260811 02:49:35.720742 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 13960
I20260811 02:49:35.732193 14755 tablet_replica.cc:330] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f: stopping tablet replica
I20260811 02:49:35.732627 14755 raft_consensus.cc:2229] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:49:35.732945 14755 raft_consensus.cc:2258] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:49:35.734040 14325 tablet_service.cc:1514] Processing DeleteTablet for tablet 2861ae426df84b59b231a55c0b4a3cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:49:35 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:35690
I20260811 02:49:35.734850 14755 ts_tablet_manager.cc:1905] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:49:35.735868 14756 tablet_replica.cc:330] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88: stopping tablet replica
I20260811 02:49:35.736197 14756 raft_consensus.cc:2229] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:49:35.736682 14756 raft_consensus.cc:2258] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:49:35.738034 14755 ts_tablet_manager.cc:1918] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:49:35.738160 14755 log.cc:1196] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/2861ae426df84b59b231a55c0b4a3cd2
I20260811 02:49:35.738431 14756 ts_tablet_manager.cc:1905] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:49:35.738591 14755 ts_tablet_manager.cc:1939] T 2861ae426df84b59b231a55c0b4a3cd2 P 4be7e38240de4b3b9c2b61aa129f4a6f: Deleting consensus metadata
I20260811 02:49:35.743350 13826 catalog_manager.cc:4841] TS 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269): tablet 2861ae426df84b59b231a55c0b4a3cd2 (table test-table [id=5a641e02304449fcb2e0a0240fac2d2e]) successfully deleted
I20260811 02:49:35.745353 14756 ts_tablet_manager.cc:1918] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:49:35.745479 14756 log.cc:1196] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/2861ae426df84b59b231a55c0b4a3cd2
W20260811 02:49:35.745584 13828 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:42999 (error 108)
I20260811 02:49:35.745798 14756 ts_tablet_manager.cc:1939] T 2861ae426df84b59b231a55c0b4a3cd2 P 1601bebbded246e3874fc777ac130b88: Deleting consensus metadata
I20260811 02:49:35.746631 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 14207
I20260811 02:49:35.752939 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.752610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42999 (local address 127.0.0.1:43650)
0811 02:49:35.752758 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:35.752849 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:42999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 02:49:35.753182 13828 catalog_manager.cc:4584] TS d1d236ed396945fea79e9f4c17bdb879 (127.21.236.129:42999): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2861ae426df84b59b231a55c0b4a3cd2: Network error: Client connection negotiation failed: client connection to 127.21.236.129:42999: connect: Connection refused (error 111)
I20260811 02:49:35.760540 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 14449
I20260811 02:49:35.760703 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:35.760414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:41397 (local address 127.0.0.1:35700)
0811 02:49:35.760562 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:35.760622 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.130:41397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 02:49:35.792806 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 13798
I20260811 02:49:35.812144 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:43417
--webserver_interface=127.21.236.190
--webserver_port=41761
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:43417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:35.926435 14759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:35.926720 14759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:35.926760 14759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:35.931363 14759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:49:35.931424 14759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:35.931445 14759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:35.931463 14759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:49:35.931478 14759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:49:35.937000 14759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:43417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:43417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=41761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:35.938056 14759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:35.939208 14759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:35.954521 14777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:35.954593 14759 server_base.cc:1034] running on GCE node
W20260811 02:49:35.954515 14768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:35.954521 14770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:35.955531 14759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:35.956719 14759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:35.957962 14759 hybrid_clock.cc:648] HybridClock initialized: now 1786416575957928 us; error 59 us; skew 500 ppm
Aug 11 02:49:35 dist-test-slave-n326 krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:35.961541 14759 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:49:35.962546 14759 webserver.cc:460] Webserver started at http://127.21.236.190:41761/ using document root <none> and password file <none>
I20260811 02:49:35.963268 14759 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:35.963395 14759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:35.968461 14759 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:49:35.970965 14787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:35.972410 14759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:49:35.972587 14759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "9b613f24f6cb47f085dcce9afd470cd5"
format_stamp: "Formatted at 2026-08-11 02:49:34 on dist-test-slave-n326"
server_key: "d2f94fdc789d82b279c4957840f2413d"
server_key_iv: "eba7e19009e3826b50dd6b9ed00a59ce"
server_key_version: "encryptionkey@0"
I20260811 02:49:35.973246 14759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:35.995489 14759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:35.998984 14759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:35.999265 14759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:36.007107 14759 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:43417
I20260811 02:49:36.007459 14894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:43417 every 8 connection(s)
I20260811 02:49:36.008315 14759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:49:36.009639 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 14759
I20260811 02:49:36.010090 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:42999
--local_ip_for_outbound_sockets=127.21.236.129
--tserver_master_addrs=127.21.236.190:43417
--webserver_port=35045
--webserver_interface=127.21.236.129
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:49:36.013861 14895 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:49:36.016878 14895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Bootstrap starting.
I20260811 02:49:36.042660 14895 log.cc:824] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:36.053925 14895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:49:36.054392 14895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Bootstrap complete.
I20260811 02:49:36.058784 14895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } }
I20260811 02:49:36.059293 14895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b613f24f6cb47f085dcce9afd470cd5, State: Initialized, Role: FOLLOWER
I20260811 02:49:36.059772 14895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } }
I20260811 02:49:36.059916 14895 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:49:36.060017 14895 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:49:36.060145 14895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:49:36.062091 14895 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } }
I20260811 02:49:36.062436 14895 leader_election.cc:302] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b613f24f6cb47f085dcce9afd470cd5; no voters:
I20260811 02:49:36.062788 14895 leader_election.cc:288] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:49:36.062916 14900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:49:36.063198 14900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 2 LEADER]: Becoming Leader. State: Replica: 9b613f24f6cb47f085dcce9afd470cd5, State: Running, Role: LEADER
I20260811 02:49:36.063529 14900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } }
I20260811 02:49:36.064026 14895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:49:36.064934 14901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9b613f24f6cb47f085dcce9afd470cd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } } }
I20260811 02:49:36.065074 14901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:36.065487 14902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b613f24f6cb47f085dcce9afd470cd5. Latest consensus state: current_term: 2 leader_uuid: "9b613f24f6cb47f085dcce9afd470cd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } } }
I20260811 02:49:36.065631 14902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:36.065943 14914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:49:36.070763 14914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:49:36.071699 14914 catalog_manager.cc:1259] Loaded cluster ID: 876742b78fe54189a0df9d683f8e01f2
I20260811 02:49:36.071765 14914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:49:36.073189 14914 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 02:49:36.078106 14914 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 02:49:36.084024 14914 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:49:36.084986 14914 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Loaded TSK: 0
I20260811 02:49:36.086239 14914 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 02:49:36.148582 14898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:36.148864 14898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:36.148908 14898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:36.153178 14898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:36.153239 14898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:36.153297 14898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:49:36.158504 14898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:42999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=35045
--tserver_master_addrs=127.21.236.190:43417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:36.159578 14898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:36.160574 14898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:36.171687 14951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:36.171675 14947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:36.171675 14945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:36.172176 14898 server_base.cc:1034] running on GCE node
I20260811 02:49:36.172644 14898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:36.173393 14898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:36.174580 14898 hybrid_clock.cc:648] HybridClock initialized: now 1786416576174561 us; error 39 us; skew 500 ppm
Aug 11 02:49:36 dist-test-slave-n326 krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416576, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:36.177675 14898 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:49:36.178896 14898 webserver.cc:460] Webserver started at http://127.21.236.129:35045/ using document root <none> and password file <none>
I20260811 02:49:36.179540 14898 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:36.179623 14898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:36.183883 14898 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:49:36.186298 14960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:36.187770 14898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:49:36.187878 14898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "d1d236ed396945fea79e9f4c17bdb879"
format_stamp: "Formatted at 2026-08-11 02:49:34 on dist-test-slave-n326"
server_key: "a769efc3966480a236c11162ccf42deb"
server_key_iv: "173c8af639d0f09917a4aea6134cf929"
server_key_version: "encryptionkey@0"
I20260811 02:49:36.188644 14898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:36.236014 14898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:36.239895 14898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:36.240128 14898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:36.240819 14898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:36.242027 14979 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 02:49:36.246316 14898 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 02:49:36.246387 14898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260811 02:49:36.246444 14898 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 02:49:36.248353 14898 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 02:49:36.248402 14898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260811 02:49:36.248801 14979 tablet_bootstrap.cc:492] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: Bootstrap starting.
I20260811 02:49:36.260218 14898 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:42999
I20260811 02:49:36.260289 15152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:42999 every 8 connection(s)
I20260811 02:49:36.261247 14898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:49:36.270213 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 14898
I20260811 02:49:36.270648 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:41397
--local_ip_for_outbound_sockets=127.21.236.130
--tserver_master_addrs=127.21.236.190:43417
--webserver_port=45147
--webserver_interface=127.21.236.130
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:49:36.272246 14979 log.cc:824] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:49:36 dist-test-slave-n326 krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416576, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:36.280642 14979 tablet_bootstrap.cc:492] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:49:36.281127 14979 tablet_bootstrap.cc:492] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: Bootstrap complete.
I20260811 02:49:36.282912 15157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.263676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:47347)
0811 02:49:36.264417 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:36.264495 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:36.265820 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:36.267235 (+ 1415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:36.267245 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:36.267825 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:36.268660 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.268683 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.270173 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.270178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.271616 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.271627 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.272831 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.272835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.273053 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.273728 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:36.273757 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:36.275788 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:36.279086 (+ 3298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:36.279093 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:36.279106 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:36.279440 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:36.279953 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:36.279956 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:36.279958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:36.280128 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:36.281006 (+ 878us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:36.281015 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:36.281307 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 02:49:36.281520 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 02:49:36.281887 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":147,"threads_started":1}
I20260811 02:49:36.283012 14979 ts_tablet_manager.cc:1397] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: Time spent bootstrapping tablet: real 0.034s user 0.029s sys 0.004s
I20260811 02:49:36.283742 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.263696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:47347 (local address 127.21.236.190:43417)
0811 02:49:36.264133 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:36.264154 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:36.264621 (+ 467us) server_negotiation.cc:407] Connection header received
0811 02:49:36.266341 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:36.266354 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:36.266735 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:36.267035 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:36.268877 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.269970 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.271784 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.272665 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.272852 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.276036 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:36.276048 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:36.276071 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:36.276192 (+ 121us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:36.278911 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:36.279603 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:36.279614 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:36.279624 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:36.279816 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:36.280263 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:36.280270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:36.280275 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:36.280848 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:36.281031 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:36.281733 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 02:49:36.282122 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":142,"threads_started":1}
I20260811 02:49:36.285561 14979 raft_consensus.cc:348] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } }
I20260811 02:49:36.286108 14979 raft_consensus.cc:729] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d236ed396945fea79e9f4c17bdb879, State: Initialized, Role: FOLLOWER
I20260811 02:49:36.286356 15153 heartbeater.cc:347] Connected to a master server at 127.21.236.190:43417
I20260811 02:49:36.286616 15153 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:36.286568 14979 consensus_queue.cc:260] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } }
I20260811 02:49:36.287235 15153 heartbeater.cc:511] Master 127.21.236.190:43417 requested a full tablet report, sending...
I20260811 02:49:36.287604 14979 ts_tablet_manager.cc:1428] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260811 02:49:36.289908 14823 ts_manager.cc:194] Registered new tserver with Master: d1d236ed396945fea79e9f4c17bdb879 (127.21.236.129:42999)
I20260811 02:49:36.293457 14823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:47347
I20260811 02:49:36.296222 15153 heartbeater.cc:503] Master 127.21.236.190:43417 was elected leader, sending a full tablet report...
Aug 11 02:49:36 dist-test-slave-n326 krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416575, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for kudu/127.21.236.129@KRBTEST.COM
I20260811 02:49:36.303838 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.292916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42999 (local address 127.0.0.1:43660)
0811 02:49:36.293672 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:36.293713 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:36.293852 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:36.294385 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:36.294391 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:36.294742 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:36.294842 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.294849 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.295873 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.295878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.296587 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.296599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.297441 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.297445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.297555 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.298156 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:36.298168 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:36.299730 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:36.302082 (+ 2352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:36.302091 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:36.302096 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:36.302324 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:36.302660 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:36.302662 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:36.302664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:36.302741 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:36.303303 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:36.303308 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:36.303412 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 02:49:36.303530 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 02:49:36.303706 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":121,"threads_started":1}
I20260811 02:49:36.304252 15166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.292987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43660 (local address 127.21.236.129:42999)
0811 02:49:36.293424 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:36.293436 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:36.293822 (+ 386us) server_negotiation.cc:407] Connection header received
0811 02:49:36.294018 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:36.294024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:36.294235 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:36.294371 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:36.294967 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.295754 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.296718 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.297298 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.297423 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.299881 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:36.299892 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:36.299905 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:36.299936 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:36.301951 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:36.302446 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:36.302453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:36.302455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:36.302560 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:36.302835 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:36.302838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:36.302840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:36.303184 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:36.303331 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:36.303648 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 02:49:36.303831 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":132,"threads_started":1}
I20260811 02:49:36.305166 15040 tablet_service.cc:1514] Processing DeleteTablet for tablet 2861ae426df84b59b231a55c0b4a3cd2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:49:35 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:43660
I20260811 02:49:36.306002 15167 tablet_replica.cc:330] stopping tablet replica
I20260811 02:49:36.306303 15167 raft_consensus.cc:2229] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:49:36.306565 15167 raft_consensus.cc:2258] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:49:36.307595 15167 ts_tablet_manager.cc:1905] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:49:36.311026 15167 ts_tablet_manager.cc:1918] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:49:36.311178 15167 log.cc:1196] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal/wals/2861ae426df84b59b231a55c0b4a3cd2
I20260811 02:49:36.311496 15167 ts_tablet_manager.cc:1939] T 2861ae426df84b59b231a55c0b4a3cd2 P d1d236ed396945fea79e9f4c17bdb879: Deleting consensus metadata
I20260811 02:49:36.313192 14793 catalog_manager.cc:4841] TS d1d236ed396945fea79e9f4c17bdb879 (127.21.236.129:42999): tablet 2861ae426df84b59b231a55c0b4a3cd2 (table test-table [id=5a641e02304449fcb2e0a0240fac2d2e]) successfully deleted
W20260811 02:49:36.399396 15161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:36.399674 15161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:36.399716 15161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:36.403923 15161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:36.403985 15161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:36.404043 15161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:49:36.409240 15161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:41397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=45147
--tserver_master_addrs=127.21.236.190:43417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:36.410243 15161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:36.411306 15161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:36.421195 15173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:36.421195 15177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:36.421257 15174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:36.421473 15161 server_base.cc:1034] running on GCE node
I20260811 02:49:36.422036 15161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:36.422688 15161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:36.423897 15161 hybrid_clock.cc:648] HybridClock initialized: now 1786416576423859 us; error 28 us; skew 500 ppm
Aug 11 02:49:36 dist-test-slave-n326 krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416576, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:36.427166 15161 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:49:36.428077 15161 webserver.cc:460] Webserver started at http://127.21.236.130:45147/ using document root <none> and password file <none>
I20260811 02:49:36.428647 15161 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:36.428728 15161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:36.433068 15161 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:49:36.435340 15188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:36.436648 15161 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:49:36.436789 15161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "1601bebbded246e3874fc777ac130b88"
format_stamp: "Formatted at 2026-08-11 02:49:35 on dist-test-slave-n326"
server_key: "a7ca90cf4d8d51665593e4acc854f1b6"
server_key_iv: "3b100dca3be3624d89409eb5063f9708"
server_key_version: "encryptionkey@0"
I20260811 02:49:36.437362 15161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:36.469758 15161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:36.473263 15161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:36.473488 15161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:36.474157 15161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:36.475046 15161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:36.475124 15161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:36.475193 15161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:36.475226 15161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:36.484848 15161 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:41397
I20260811 02:49:36.484880 15365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:41397 every 8 connection(s)
I20260811 02:49:36.485872 15161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:49:36.490015 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 15161
I20260811 02:49:36.490443 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:43269
--local_ip_for_outbound_sockets=127.21.236.131
--tserver_master_addrs=127.21.236.190:43417
--webserver_port=45923
--webserver_interface=127.21.236.131
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:49:36 dist-test-slave-n326 krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416576, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:36.500869 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.487765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:47773 (local address 127.21.236.190:43417)
0811 02:49:36.487908 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:36.487912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:36.488703 (+ 791us) server_negotiation.cc:407] Connection header received
0811 02:49:36.490086 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:36.490090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:36.490148 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:36.490253 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:36.491899 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.492581 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.493903 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.494147 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.494261 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.497167 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:36.497175 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:36.497177 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:36.497201 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:36.498670 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:36.499250 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:36.499255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:36.499257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:36.499340 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:36.499706 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:36.499709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:36.499711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:36.499895 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:36.500009 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:36.500551 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 02:49:36.500689 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:49:36.501893 15369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.488105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:47773)
0811 02:49:36.488554 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:36.488596 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:36.489866 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:36.490427 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:36.490434 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:36.490980 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:36.491744 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.491761 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.492710 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.492713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.493776 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.493787 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.494303 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.494307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.494484 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.495196 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:36.495223 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:36.496986 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:36.498795 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:36.498800 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:36.498815 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:36.499140 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:36.499438 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:36.499442 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:36.499444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:36.499598 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:36.500006 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:36.500013 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:36.500303 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 02:49:36.500489 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 02:49:36.500827 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":119,"threads_started":1}
I20260811 02:49:36.503239 15367 heartbeater.cc:347] Connected to a master server at 127.21.236.190:43417
I20260811 02:49:36.503541 15367 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:36.504108 15367 heartbeater.cc:511] Master 127.21.236.190:43417 requested a full tablet report, sending...
I20260811 02:49:36.505151 14823 ts_manager.cc:194] Registered new tserver with Master: 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397)
I20260811 02:49:36.505900 14823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:47773
W20260811 02:49:36.608814 15371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:36.609067 15371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:36.609108 15371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:36.613050 15371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:36.613109 15371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:36.613165 15371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:49:36.618393 15371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:43269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=45923
--tserver_master_addrs=127.21.236.190:43417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:36.619414 15371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:36.620338 15371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:49:36.630049 15371 server_base.cc:1034] running on GCE node
W20260811 02:49:36.630065 15377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:36.630066 15378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:36.630076 15380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:36.630780 15371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:36.631470 15371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:36.632658 15371 hybrid_clock.cc:648] HybridClock initialized: now 1786416576632628 us; error 44 us; skew 500 ppm
Aug 11 02:49:36 dist-test-slave-n326 krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416576, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:36.635560 15371 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:49:36.636453 15371 webserver.cc:460] Webserver started at http://127.21.236.131:45923/ using document root <none> and password file <none>
I20260811 02:49:36.636979 15371 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:36.637048 15371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:36.641474 15371 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:49:36.643697 15390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:36.644907 15371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:49:36.645015 15371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "4be7e38240de4b3b9c2b61aa129f4a6f"
format_stamp: "Formatted at 2026-08-11 02:49:35 on dist-test-slave-n326"
server_key: "2613966eb5a013778c94a6d4bef3ed60"
server_key_iv: "5fc8166c9d157303447b30782a1f3bfb"
server_key_version: "encryptionkey@0"
I20260811 02:49:36.645509 15371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:36.660998 15371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:36.664271 15371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:36.664424 15371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:36.665109 15371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:36.665984 15371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:36.666033 15371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:36.666072 15371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:36.666115 15371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:36.675467 15371 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:43269
I20260811 02:49:36.675506 15598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:43269 every 8 connection(s)
I20260811 02:49:36.676527 15371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:49:36.678172 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 15371
Aug 11 02:49:36 dist-test-slave-n326 krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416576, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:36.688167 15605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.678833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.21.236.190:43417)
0811 02:49:36.679058 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:36.679103 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:36.679387 (+ 284us) server_negotiation.cc:407] Connection header received
0811 02:49:36.679522 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:36.679526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:36.679619 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:36.679715 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:36.680879 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.681578 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.682755 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.683131 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.683229 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.684644 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:36.684648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:36.684650 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:36.684678 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:36.686338 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:36.686781 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:36.686786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:36.686788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:36.686874 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:36.687191 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:36.687195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:36.687196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:36.687411 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:36.687628 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:36.687837 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 02:49:36.687969 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
I20260811 02:49:36.691346 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.678412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:36035 (local address 127.21.236.190:43417)
0811 02:49:36.678602 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:36.678606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:36.679699 (+ 1093us) server_negotiation.cc:407] Connection header received
0811 02:49:36.681382 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:36.681385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:36.681428 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:36.681512 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:36.682908 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.683580 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.684755 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.685082 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.685206 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.687703 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:36.687707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:36.687709 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:36.687733 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:36.689343 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:36.690006 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:36.690012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:36.690016 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:36.690113 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:36.690403 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:36.690405 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:36.690406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:36.690545 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:36.690614 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:36.691134 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 02:49:36.691235 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":61}
I20260811 02:49:36.692252 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.678942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:36035)
0811 02:49:36.679522 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:36.679577 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:36.681174 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:36.681665 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:36.681673 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:36.682153 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:36.682773 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.682792 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.683705 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.683712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.684644 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.684654 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.685265 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.685268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.685415 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.686000 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:36.686016 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:36.687539 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:36.689464 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:36.689469 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:36.689483 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:36.689774 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:36.690200 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:36.690202 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:36.690204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:36.690334 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:36.690643 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:36.690648 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:36.690867 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 02:49:36.691017 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 02:49:36.691337 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":151,"threads_started":1}
I20260811 02:49:36.693367 15600 heartbeater.cc:347] Connected to a master server at 127.21.236.190:43417
I20260811 02:49:36.693589 15600 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:36.694011 15600 heartbeater.cc:511] Master 127.21.236.190:43417 requested a full tablet report, sending...
I20260811 02:49:36.694959 14823 ts_manager.cc:194] Registered new tserver with Master: 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269)
I20260811 02:49:36.695533 14823 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:36035
I20260811 02:49:36.701653 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:49:36.705569 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.702216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47124 (local address 127.21.236.190:43417)
0811 02:49:36.702308 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:36.702312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:36.702552 (+ 240us) server_negotiation.cc:407] Connection header received
0811 02:49:36.702660 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:36.702662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:36.702746 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:36.702825 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:49:36.703030 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.703567 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.704392 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.704595 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.704669 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.704946 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:49:36.705295 (+ 349us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:49:36.705381 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:36.705436 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 02:49:36.705476 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 02:49:36.706182 14823 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:49:36.709444 14823 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:49:36.717260 15040 tablet_service.cc:1467] Processing CreateTablet for tablet 184b94869cdc4b4a9af3614448a77f75 (DEFAULT_TABLE table=test-table [id=a1ebb7952d624eb99f8c39448952a000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:49:36.717962 15040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 184b94869cdc4b4a9af3614448a77f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:36.720700 14979 tablet_bootstrap.cc:492] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879: Bootstrap starting.
I20260811 02:49:36.721500 14979 tablet_bootstrap.cc:654] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:36.722410 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.716177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:41397 (local address 127.0.0.1:52756)
0811 02:49:36.716284 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:36.716301 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:36.716376 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:36.717558 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:36.717561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:36.717590 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:36.717709 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.717715 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.719559 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.719563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.720680 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.720689 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.721937 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.721941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.722058 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.722081 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:49:36.722165 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 02:49:36.722246 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":27}
I20260811 02:49:36.722429 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.716777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:43269 (local address 127.0.0.1:33972)
0811 02:49:36.717081 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:36.717095 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:36.717180 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:36.717832 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:36.717835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:36.717853 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:36.717951 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.717958 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.719279 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.719284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.720528 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.720539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.721842 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.721847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.721963 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.722025 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 02:49:36.722129 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 02:49:36.722213 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":1,"thread_start_us":90,"threads_started":1}
I20260811 02:49:36.722909 15627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.716935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33972 (local address 127.21.236.131:43269)
0811 02:49:36.717396 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:36.717405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:36.717422 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:49:36.717517 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:36.717522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:36.717697 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:36.717862 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:36.718061 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.719153 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.720681 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.721692 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.721825 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.721986 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:36.722184 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 02:49:36.722280 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":100,"threads_started":1}
I20260811 02:49:36.723140 15624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.716532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52756 (local address 127.21.236.130:41397)
0811 02:49:36.717032 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:36.717044 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:36.717069 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:49:36.717151 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:36.717157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:36.717377 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:36.717870 (+ 493us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:36.718037 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.719414 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.720986 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.721805 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.722646 (+ 841us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.722788 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:36.722863 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 02:49:36.722963 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":173,"threads_started":1}
I20260811 02:49:36.724639 14979 tablet_bootstrap.cc:492] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879: No bootstrap required, opened a new log
I20260811 02:49:36.724742 14979 ts_tablet_manager.cc:1397] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 02:49:36.725220 14979 raft_consensus.cc:348] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:36.725355 14979 raft_consensus.cc:374] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:36.725396 14979 raft_consensus.cc:729] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d236ed396945fea79e9f4c17bdb879, State: Initialized, Role: FOLLOWER
I20260811 02:49:36.725531 14979 consensus_queue.cc:260] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:36.725811 14979 ts_tablet_manager.cc:1428] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 02:49:36.725862 15481 tablet_service.cc:1467] Processing CreateTablet for tablet 184b94869cdc4b4a9af3614448a77f75 (DEFAULT_TABLE table=test-table [id=a1ebb7952d624eb99f8c39448952a000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:49:36.725863 15259 tablet_service.cc:1467] Processing CreateTablet for tablet 184b94869cdc4b4a9af3614448a77f75 (DEFAULT_TABLE table=test-table [id=a1ebb7952d624eb99f8c39448952a000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:49:36.727016 15259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 184b94869cdc4b4a9af3614448a77f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:36.727056 15481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 184b94869cdc4b4a9af3614448a77f75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:36.733974 15631 tablet_bootstrap.cc:492] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88: Bootstrap starting.
I20260811 02:49:36.734020 15630 tablet_bootstrap.cc:492] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f: Bootstrap starting.
I20260811 02:49:36.736651 15630 tablet_bootstrap.cc:654] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:36.736651 15631 tablet_bootstrap.cc:654] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:36.737437 15630 log.cc:824] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:36.737437 15631 log.cc:824] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:36.739303 15631 tablet_bootstrap.cc:492] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88: No bootstrap required, opened a new log
I20260811 02:49:36.739310 15630 tablet_bootstrap.cc:492] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f: No bootstrap required, opened a new log
I20260811 02:49:36.739502 15630 ts_tablet_manager.cc:1397] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:49:36.739502 15631 ts_tablet_manager.cc:1397] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:49:36.742636 15630 raft_consensus.cc:348] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:36.742883 15630 raft_consensus.cc:374] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:36.742980 15630 raft_consensus.cc:729] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4be7e38240de4b3b9c2b61aa129f4a6f, State: Initialized, Role: FOLLOWER
I20260811 02:49:36.742851 15631 raft_consensus.cc:348] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:36.743147 15631 raft_consensus.cc:374] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:36.743208 15631 raft_consensus.cc:729] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1601bebbded246e3874fc777ac130b88, State: Initialized, Role: FOLLOWER
I20260811 02:49:36.743487 15630 consensus_queue.cc:260] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:36.743631 15631 consensus_queue.cc:260] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:36.744192 15600 heartbeater.cc:503] Master 127.21.236.190:43417 was elected leader, sending a full tablet report...
I20260811 02:49:36.744319 15367 heartbeater.cc:503] Master 127.21.236.190:43417 was elected leader, sending a full tablet report...
I20260811 02:49:36.744580 15630 ts_tablet_manager.cc:1428] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:49:36.744606 15631 ts_tablet_manager.cc:1428] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260811 02:49:36.762538 15154 tablet.cc:2369] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:49:36.815563 15163 raft_consensus.cc:482] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:49:36.815783 15163 raft_consensus.cc:504] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:36.816897 15163 leader_election.cc:288] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397), 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269)
I20260811 02:49:36.820989 15624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.817231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:33947 (local address 127.21.236.130:41397)
0811 02:49:36.817406 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:36.817411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:36.817542 (+ 131us) server_negotiation.cc:407] Connection header received
0811 02:49:36.817792 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:36.817795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:36.817860 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:36.817967 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:36.818245 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.818745 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.819967 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.820399 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.820536 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.820588 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:36.820800 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 02:49:36.820867 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 02:49:36.821003 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.817079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:41397 (local address 127.21.236.129:33947)
0811 02:49:36.817423 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:36.817442 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:36.817574 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:36.817979 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:36.817982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:36.817998 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:36.818133 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.818139 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.818867 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.818872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.819838 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.819849 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.820530 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.820534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.820653 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.820690 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 02:49:36.820784 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 02:49:36.820859 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":53,"thread_start_us":96,"threads_started":1}
I20260811 02:49:36.821146 15627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.817309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:46503 (local address 127.21.236.131:43269)
0811 02:49:36.817426 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:36.817431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:36.817703 (+ 272us) server_negotiation.cc:407] Connection header received
0811 02:49:36.817929 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:36.817932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:36.817983 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:36.818079 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:36.818370 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.818940 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.820095 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.820541 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.820680 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.820734 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:36.820930 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 02:49:36.821005 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:49:36.821173 15638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.817196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:43269 (local address 127.21.236.129:46503)
0811 02:49:36.817613 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:36.817625 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:36.817742 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:36.818106 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:36.818110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:36.818129 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:36.818238 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.818246 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.819091 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.819096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.819938 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:36.819948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.820680 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.820684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:36.820792 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.820818 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 02:49:36.820913 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 02:49:36.821000 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":103,"threads_started":1}
I20260811 02:49:36.821830 15294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "184b94869cdc4b4a9af3614448a77f75" candidate_uuid: "d1d236ed396945fea79e9f4c17bdb879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1601bebbded246e3874fc777ac130b88" is_pre_election: true
I20260811 02:49:36.821902 15512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "184b94869cdc4b4a9af3614448a77f75" candidate_uuid: "d1d236ed396945fea79e9f4c17bdb879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" is_pre_election: true
I20260811 02:49:36.822178 15294 raft_consensus.cc:2454] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1d236ed396945fea79e9f4c17bdb879 in term 0.
I20260811 02:49:36.822220 15512 raft_consensus.cc:2454] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1d236ed396945fea79e9f4c17bdb879 in term 0.
I20260811 02:49:36.822710 14975 leader_election.cc:302] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1601bebbded246e3874fc777ac130b88, d1d236ed396945fea79e9f4c17bdb879; no voters:
I20260811 02:49:36.823017 15163 raft_consensus.cc:2781] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:49:36.823137 15163 raft_consensus.cc:482] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:49:36.823197 15163 raft_consensus.cc:3037] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:36.824218 15163 raft_consensus.cc:504] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:36.824570 15163 leader_election.cc:288] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 1 election: Requested vote from peers 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397), 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269)
I20260811 02:49:36.824913 15294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "184b94869cdc4b4a9af3614448a77f75" candidate_uuid: "d1d236ed396945fea79e9f4c17bdb879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1601bebbded246e3874fc777ac130b88"
I20260811 02:49:36.825033 15294 raft_consensus.cc:3037] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:36.825045 15512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "184b94869cdc4b4a9af3614448a77f75" candidate_uuid: "d1d236ed396945fea79e9f4c17bdb879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f"
I20260811 02:49:36.825205 15512 raft_consensus.cc:3037] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:36.826014 15294 raft_consensus.cc:2454] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1d236ed396945fea79e9f4c17bdb879 in term 1.
I20260811 02:49:36.826325 14975 leader_election.cc:302] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1601bebbded246e3874fc777ac130b88, d1d236ed396945fea79e9f4c17bdb879; no voters:
I20260811 02:49:36.826501 15163 raft_consensus.cc:2781] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:49:36.826607 15512 raft_consensus.cc:2454] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1d236ed396945fea79e9f4c17bdb879 in term 1.
I20260811 02:49:36.826680 15163 raft_consensus.cc:686] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 LEADER]: Becoming Leader. State: Replica: d1d236ed396945fea79e9f4c17bdb879, State: Running, Role: LEADER
I20260811 02:49:36.826969 15163 consensus_queue.cc:237] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:36.829946 14823 catalog_manager.cc:5495] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1d236ed396945fea79e9f4c17bdb879 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "d1d236ed396945fea79e9f4c17bdb879" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } health_report { overall_health: HEALTHY } } }
I20260811 02:49:36.847513 15644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.843737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34590 (local address 127.21.236.129:42999)
0811 02:49:36.844040 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:36.844050 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:36.844067 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:49:36.844144 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:36.844146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:36.844199 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:36.844331 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:49:36.844592 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.845116 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.845969 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:36.846209 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:36.846324 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:36.846611 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:49:36.847027 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:49:36.847232 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:36.847294 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 02:49:36.847351 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":103,"threads_started":1}
I20260811 02:49:36.851584 15512 raft_consensus.cc:1261] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 1 FOLLOWER]: Refusing update from remote peer d1d236ed396945fea79e9f4c17bdb879: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:49:36.851718 15294 raft_consensus.cc:1261] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88 [term 1 FOLLOWER]: Refusing update from remote peer d1d236ed396945fea79e9f4c17bdb879: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:49:36.852409 15640 consensus_queue.cc:1035] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:49:36.852651 15163 consensus_queue.cc:1035] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:49:36.869333 14823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:49:36.869519 14823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:49:36.871905 14823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Sending DeleteTablet for 3 replicas of tablet 184b94869cdc4b4a9af3614448a77f75
I20260811 02:49:36.872656 15259 tablet_service.cc:1514] Processing DeleteTablet for tablet 184b94869cdc4b4a9af3614448a77f75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:49:36 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:52756
I20260811 02:49:36.872656 15481 tablet_service.cc:1514] Processing DeleteTablet for tablet 184b94869cdc4b4a9af3614448a77f75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:49:36 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:33972
I20260811 02:49:36.872808 15040 tablet_service.cc:1514] Processing DeleteTablet for tablet 184b94869cdc4b4a9af3614448a77f75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:49:36 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:43660
I20260811 02:49:36.873234 15661 tablet_replica.cc:330] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879: stopping tablet replica
I20260811 02:49:36.873417 15661 raft_consensus.cc:2229] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:49:36.873423 15657 tablet_replica.cc:330] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f: stopping tablet replica
I20260811 02:49:36.873436 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 14898
I20260811 02:49:36.887230 15657 raft_consensus.cc:2229] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:49:36.887785 15657 raft_consensus.cc:2258] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:49:36.888383 15659 tablet_replica.cc:330] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88: stopping tablet replica
I20260811 02:49:36.888825 15659 raft_consensus.cc:2229] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:49:36.889235 15659 raft_consensus.cc:2258] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:49:36.892475 15659 ts_tablet_manager.cc:1905] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:49:36.896715 15659 ts_tablet_manager.cc:1918] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:49:36.896858 15659 log.cc:1196] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/184b94869cdc4b4a9af3614448a77f75
I20260811 02:49:36.897295 15659 ts_tablet_manager.cc:1939] T 184b94869cdc4b4a9af3614448a77f75 P 1601bebbded246e3874fc777ac130b88: Deleting consensus metadata
I20260811 02:49:36.899394 14796 catalog_manager.cc:4841] TS 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397): tablet 184b94869cdc4b4a9af3614448a77f75 (table test-table [id=a1ebb7952d624eb99f8c39448952a000]) successfully deleted
I20260811 02:49:36.903021 15657 ts_tablet_manager.cc:1905] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:49:36.905308 15657 ts_tablet_manager.cc:1918] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:49:36.905400 15657 log.cc:1196] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/184b94869cdc4b4a9af3614448a77f75
I20260811 02:49:36.905746 15657 ts_tablet_manager.cc:1939] T 184b94869cdc4b4a9af3614448a77f75 P 4be7e38240de4b3b9c2b61aa129f4a6f: Deleting consensus metadata
I20260811 02:49:36.906960 14789 catalog_manager.cc:4841] TS 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269): tablet 184b94869cdc4b4a9af3614448a77f75 (table test-table [id=a1ebb7952d624eb99f8c39448952a000]) successfully deleted
W20260811 02:49:36.909210 14793 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:42999 (error 108)
I20260811 02:49:36.910099 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 15161
I20260811 02:49:36.910359 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:36.910017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42999 (local address 127.0.0.1:34604)
0811 02:49:36.910167 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:36.910259 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:42999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 02:49:36.917481 14793 catalog_manager.cc:4584] TS d1d236ed396945fea79e9f4c17bdb879 (127.21.236.129:42999): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 184b94869cdc4b4a9af3614448a77f75: Network error: Client connection negotiation failed: client connection to 127.21.236.129:42999: connect: Connection refused (error 111)
I20260811 02:49:36.943177 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 15371
I20260811 02:49:36.952703 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 14759
I20260811 02:49:36.964764 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:43417
--webserver_interface=127.21.236.190
--webserver_port=41761
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:43417
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:37.077677 15664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:37.077941 15664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:37.077981 15664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:37.081981 15664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:49:37.082039 15664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:37.082058 15664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:37.082075 15664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:49:37.082090 15664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:49:37.087359 15664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:43417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:43417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=41761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:37.088452 15664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:37.089540 15664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:37.098337 15672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:37.098376 15677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:37.098523 15664 server_base.cc:1034] running on GCE node
W20260811 02:49:37.098337 15673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:37.099248 15664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:37.100462 15664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:37.101662 15664 hybrid_clock.cc:648] HybridClock initialized: now 1786416577101645 us; error 31 us; skew 500 ppm
Aug 11 02:49:37 dist-test-slave-n326 krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416577, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:37.104592 15664 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:49:37.105517 15664 webserver.cc:460] Webserver started at http://127.21.236.190:41761/ using document root <none> and password file <none>
I20260811 02:49:37.106113 15664 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:37.106213 15664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:37.110306 15664 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:49:37.112653 15687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:37.113968 15664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:49:37.114123 15664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "9b613f24f6cb47f085dcce9afd470cd5"
format_stamp: "Formatted at 2026-08-11 02:49:34 on dist-test-slave-n326"
server_key: "d2f94fdc789d82b279c4957840f2413d"
server_key_iv: "eba7e19009e3826b50dd6b9ed00a59ce"
server_key_version: "encryptionkey@0"
I20260811 02:49:37.114702 15664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:37.125600 15664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:37.128795 15664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:37.129009 15664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:37.136550 15664 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:43417
I20260811 02:49:37.136873 15792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:43417 every 8 connection(s)
I20260811 02:49:37.137694 15664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:49:37.141980 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 15664
I20260811 02:49:37.142503 15795 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:49:37.142535 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:42999
--local_ip_for_outbound_sockets=127.21.236.129
--tserver_master_addrs=127.21.236.190:43417
--webserver_port=35045
--webserver_interface=127.21.236.129
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:49:37.145126 15795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Bootstrap starting.
I20260811 02:49:37.164511 15795 log.cc:824] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:37.177008 15795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:49:37.177445 15795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5: Bootstrap complete.
I20260811 02:49:37.182008 15795 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } }
I20260811 02:49:37.182410 15795 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b613f24f6cb47f085dcce9afd470cd5, State: Initialized, Role: FOLLOWER
I20260811 02:49:37.182933 15795 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } }
I20260811 02:49:37.183109 15795 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:49:37.183187 15795 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:49:37.183313 15795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:49:37.185364 15795 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } }
I20260811 02:49:37.185720 15795 leader_election.cc:302] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b613f24f6cb47f085dcce9afd470cd5; no voters:
I20260811 02:49:37.186081 15795 leader_election.cc:288] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 02:49:37.186180 15802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:49:37.186408 15802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [term 3 LEADER]: Becoming Leader. State: Replica: 9b613f24f6cb47f085dcce9afd470cd5, State: Running, Role: LEADER
I20260811 02:49:37.186738 15802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } }
I20260811 02:49:37.187465 15795 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:49:37.188270 15803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9b613f24f6cb47f085dcce9afd470cd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } } }
I20260811 02:49:37.188405 15803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:37.189229 15804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b613f24f6cb47f085dcce9afd470cd5. Latest consensus state: current_term: 3 leader_uuid: "9b613f24f6cb47f085dcce9afd470cd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b613f24f6cb47f085dcce9afd470cd5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43417 } } }
I20260811 02:49:37.189383 15804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b613f24f6cb47f085dcce9afd470cd5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:37.189754 15818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:49:37.195120 15818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:49:37.195730 15818 catalog_manager.cc:1259] Loaded cluster ID: 876742b78fe54189a0df9d683f8e01f2
I20260811 02:49:37.195793 15818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 02:49:37.196849 15818 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa0db4839ad google::LogMessage::Fail() at ??:0
@ 0x7fa0db48688a google::LogMessage::SendToLog() at ??:0
@ 0x7fa0db4834de google::LogMessage::Flush() at ??:0
@ 0x7fa0db484a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa0e5dc2fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fa0e5dc3d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa0e5dc0579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fa0e5e04640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fa0dd6d53d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa0dd6d5c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fa0dd6d73f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fa0dd6c6512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa0ded0a6db start_thread at ??:0
@ 0x7fa0d9f8161f clone at ??:0
W20260811 02:49:37.279851 15800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:37.280134 15800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:37.280244 15800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:37.284322 15800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:37.284405 15800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:37.284516 15800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:49:37.289963 15800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:42999
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=35045
--tserver_master_addrs=127.21.236.190:43417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:37.291143 15800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:37.292210 15800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:37.302130 15873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:37.302138 15878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:37.302172 15874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:37.302505 15800 server_base.cc:1034] running on GCE node
I20260811 02:49:37.302940 15800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:37.303620 15800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:37.304817 15800 hybrid_clock.cc:648] HybridClock initialized: now 1786416577304776 us; error 57 us; skew 500 ppm
Aug 11 02:49:37 dist-test-slave-n326 krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416577, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:37.307684 15800 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:49:37.308661 15800 webserver.cc:460] Webserver started at http://127.21.236.129:35045/ using document root <none> and password file <none>
I20260811 02:49:37.309249 15800 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:37.309345 15800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:37.313395 15800 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 02:49:37.316138 15900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:37.317481 15800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:49:37.317616 15800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "d1d236ed396945fea79e9f4c17bdb879"
format_stamp: "Formatted at 2026-08-11 02:49:34 on dist-test-slave-n326"
server_key: "a769efc3966480a236c11162ccf42deb"
server_key_iv: "173c8af639d0f09917a4aea6134cf929"
server_key_version: "encryptionkey@0"
I20260811 02:49:37.318177 15800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:37.338217 15800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:37.341593 15800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:37.341805 15800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:37.342434 15800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:37.343751 15912 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 02:49:37.349377 15800 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 02:49:37.349465 15800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 02:49:37.349577 15800 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 02:49:37.351294 15800 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 02:49:37.351364 15800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260811 02:49:37.351671 15912 tablet_bootstrap.cc:492] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879: Bootstrap starting.
I20260811 02:49:37.362466 15800 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:42999
I20260811 02:49:37.362510 16106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:42999 every 8 connection(s)
I20260811 02:49:37.363559 15800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:49:37.370980 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 15800
I20260811 02:49:37.371430 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:41397
--local_ip_for_outbound_sockets=127.21.236.130
--tserver_master_addrs=127.21.236.190:43417
--webserver_port=45147
--webserver_interface=127.21.236.130
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:49:37.374476 15912 log.cc:824] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:37.384079 15912 tablet_bootstrap.cc:492] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:49:37.384538 15912 tablet_bootstrap.cc:492] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879: Bootstrap complete.
I20260811 02:49:37.386186 15912 ts_tablet_manager.cc:1397] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879: Time spent bootstrapping tablet: real 0.035s user 0.016s sys 0.015s
I20260811 02:49:37.389595 15912 raft_consensus.cc:348] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:37.390179 15912 raft_consensus.cc:729] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d236ed396945fea79e9f4c17bdb879, State: Initialized, Role: FOLLOWER
I20260811 02:49:37.390738 15912 consensus_queue.cc:260] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:37.391863 15912 ts_tablet_manager.cc:1428] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260811 02:49:37.435925 16109 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:49:37.365523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:40791)
0811 02:49:37.366016 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:37.366081 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:37.366894 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:37.434893 (+ 67999us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":144,"threads_started":1}
W20260811 02:49:37.436578 16107 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:43417 (0 consecutive failures): Network error: Failed to ping master at 127.21.236.190:43417: Client connection negotiation failed: client connection to 127.21.236.190:43417: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 02:49:37.437371 16109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:37.437058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:47827)
0811 02:49:37.437209 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:37.437299 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 02:49:37.438094 16109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:37.437841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:54785)
0811 02:49:37.437968 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:37.438020 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 02:49:37.438308 16107 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:49:37.489737 16110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:37.489992 16110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:37.490034 16110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:37.494071 16110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:37.494132 16110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:37.494189 16110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:49:37.499437 16110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:41397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=45147
--tserver_master_addrs=127.21.236.190:43417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:37.500489 16110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:37.501586 16110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:37.511518 16134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:37.511533 16127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:37.511581 16110 server_base.cc:1034] running on GCE node
W20260811 02:49:37.511518 16123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:37.512161 16110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:37.512779 16110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:37.513949 16110 hybrid_clock.cc:648] HybridClock initialized: now 1786416577513929 us; error 32 us; skew 500 ppm
Aug 11 02:49:37 dist-test-slave-n326 krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416577, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:37.516774 16110 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:49:37.517674 16110 webserver.cc:460] Webserver started at http://127.21.236.130:45147/ using document root <none> and password file <none>
I20260811 02:49:37.518234 16110 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:37.518296 16110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:37.522332 16110 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:49:37.524513 16145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:37.525745 16110 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:49:37.525872 16110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "1601bebbded246e3874fc777ac130b88"
format_stamp: "Formatted at 2026-08-11 02:49:35 on dist-test-slave-n326"
server_key: "a7ca90cf4d8d51665593e4acc854f1b6"
server_key_iv: "3b100dca3be3624d89409eb5063f9708"
server_key_version: "encryptionkey@0"
I20260811 02:49:37.526407 16110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:37.542500 16110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:37.545444 16110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:37.545644 16110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:37.546236 16110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:37.547052 16110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:37.547161 16110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:37.547228 16110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:37.547263 16110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:37.556628 16110 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:41397
I20260811 02:49:37.556667 16347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:41397 every 8 connection(s)
I20260811 02:49:37.557610 16110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:49:37.561192 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:37.559670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:40293)
0811 02:49:37.560107 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:37.560517 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":116,"threads_started":1}
I20260811 02:49:37.561398 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 16110
W20260811 02:49:37.561743 16348 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:43417 (0 consecutive failures): Network error: Failed to ping master at 127.21.236.190:43417: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
I20260811 02:49:37.561883 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:43269
--local_ip_for_outbound_sockets=127.21.236.131
--tserver_master_addrs=127.21.236.190:43417
--webserver_port=45923
--webserver_interface=127.21.236.131
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:49:37.562373 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:37.562171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:33837)
0811 02:49:37.562277 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:37.562322 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 02:49:37.563105 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:37.562841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:41191)
0811 02:49:37.562951 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:37.563004 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 02:49:37.563336 16348 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:49:37.677104 16354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:37.677394 16354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:37.677441 16354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:37.681663 16354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:37.681733 16354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:37.681826 16354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:49:37.687245 16354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:43269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=45923
--tserver_master_addrs=127.21.236.190:43417
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:37.688452 16354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:37.689603 16354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:37.699862 16359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:37.699851 16360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:37.699841 16362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:37.700167 16354 server_base.cc:1034] running on GCE node
I20260811 02:49:37.700711 16354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:37.701503 16354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:37.702682 16354 hybrid_clock.cc:648] HybridClock initialized: now 1786416577702663 us; error 27 us; skew 500 ppm
Aug 11 02:49:37 dist-test-slave-n326 krb5kdc[13772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416577, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:37.705600 16354 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:49:37.706550 16354 webserver.cc:460] Webserver started at http://127.21.236.131:45923/ using document root <none> and password file <none>
I20260811 02:49:37.707273 16354 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:37.707376 16354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:37.711575 16354 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:49:37.713795 16373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:37.715173 16354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:49:37.715313 16354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "4be7e38240de4b3b9c2b61aa129f4a6f"
format_stamp: "Formatted at 2026-08-11 02:49:35 on dist-test-slave-n326"
server_key: "2613966eb5a013778c94a6d4bef3ed60"
server_key_iv: "5fc8166c9d157303447b30782a1f3bfb"
server_key_version: "encryptionkey@0"
I20260811 02:49:37.715901 16354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:37.732491 16354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:37.735934 16354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:37.736191 16354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:37.736901 16354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:37.737879 16354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:37.737927 16354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:37.737996 16354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:37.738030 16354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:37.747798 16354 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:43269
I20260811 02:49:37.747818 16561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:43269 every 8 connection(s)
I20260811 02:49:37.748960 16354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:49:37.749926 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 16354
I20260811 02:49:37.753091 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:37.751478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:37267)
0811 02:49:37.751949 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:37.752370 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":121,"threads_started":1}
W20260811 02:49:37.753607 16562 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:43417 (0 consecutive failures): Network error: Failed to ping master at 127.21.236.190:43417: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
I20260811 02:49:37.754208 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:37.754002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:59193)
0811 02:49:37.754107 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:37.754154 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 02:49:37.754818 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:37.754639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:54661)
0811 02:49:37.754734 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:37.754774 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 02:49:37.755028 16562 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 02:49:38.439517 16573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:38.439012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:57437)
0811 02:49:38.439339 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:38.439401 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":86,"threads_started":1}
I20260811 02:49:38.564355 16574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:38.563920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:57647)
0811 02:49:38.564193 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:38.564248 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":80,"threads_started":1}
I20260811 02:49:38.756975 16575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:38.756519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:51459)
0811 02:49:38.756765 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:38.756833 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
I20260811 02:49:39.022598 16576 raft_consensus.cc:482] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:49:39.022861 16576 raft_consensus.cc:504] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:39.024072 16576 leader_election.cc:288] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397), 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269)
Aug 11 02:49:39 dist-test-slave-n326 krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416577, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.131@KRBTEST.COM
Aug 11 02:49:39 dist-test-slave-n326 krb5kdc[13772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416577, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.130@KRBTEST.COM
I20260811 02:49:39.044219 16582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:39.024348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:43269 (local address 127.21.236.129:47189)
0811 02:49:39.024753 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:39.024768 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:39.024914 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:39.026472 (+ 1558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:39.026478 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:39.027109 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:39.027581 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:39.027597 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:39.029115 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:39.029119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:39.030683 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:39.030695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:39.032017 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:39.032023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:39.032157 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:39.036661 (+ 4504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:39.036687 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:39.038736 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:39.041592 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:39.041597 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:39.041609 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:39.041860 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:39.042346 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:39.042350 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:39.042352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:39.042532 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:39.043216 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:39.043223 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:39.043469 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 02:49:39.043661 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 02:49:39.044000 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":200,"spinlock_wait_cycles":54656,"thread_start_us":82,"threads_started":1}
I20260811 02:49:39.044238 16584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:39.024522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:47189 (local address 127.21.236.131:43269)
0811 02:49:39.024975 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:39.024996 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:39.025036 (+ 40us) server_negotiation.cc:407] Connection header received
0811 02:49:39.025731 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:39.025737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:39.026090 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:39.026400 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:39.027763 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:39.028895 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:39.031140 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:39.031838 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:39.032022 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:39.039009 (+ 6987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:39.039015 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:39.039027 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:39.039107 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:39.041448 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:39.042107 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:39.042120 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:39.042124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:39.042242 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:39.042635 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:39.042640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:39.042642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:39.043052 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:39.043240 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:39.043770 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 02:49:39.044035 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":119,"threads_started":1}
I20260811 02:49:39.044270 16583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:39.024480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:33785 (local address 127.21.236.130:41397)
0811 02:49:39.025154 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:39.025175 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:39.025208 (+ 33us) server_negotiation.cc:407] Connection header received
0811 02:49:39.025820 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:39.025825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:39.026120 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:39.026405 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:39.027762 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:39.028937 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:39.030822 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:39.031562 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:39.031728 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:39.039600 (+ 7872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:39.039604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:39.039636 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:39.039677 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:39.041779 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:39.042220 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:39.042225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:39.042236 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:39.042346 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:39.042640 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:39.042643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:39.042645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:39.043053 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:39.043239 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:39.043786 (+ 547us) server_negotiation.cc:299] Negotiation successful
0811 02:49:39.044079 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":162,"threads_started":1}
I20260811 02:49:39.044257 16577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:39.024347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:41397 (local address 127.21.236.129:33785)
0811 02:49:39.024790 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:39.024806 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:39.024914 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:39.026472 (+ 1558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:39.026478 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:39.027120 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:39.027581 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:39.027596 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:39.029115 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:39.029119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:39.030684 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:39.030695 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:39.031702 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:39.031705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:39.031845 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:39.036684 (+ 4839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:39.036697 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:39.039368 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:39.041905 (+ 2537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:39.041908 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:39.041910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:39.042102 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:39.042446 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:39.042451 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:39.042453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:39.042541 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:39.043220 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:39.043223 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:39.043470 (+ 247us) client_negotiation.cc:769] Sending connection context
0811 02:49:39.043662 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 02:49:39.044087 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":395,"spinlock_wait_cycles":24064,"thread_start_us":50,"threads_started":1}
W20260811 02:49:39.047015 15906 leader_election.cc:341] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397): Not found: Tablet not found: 184b94869cdc4b4a9af3614448a77f75
W20260811 02:49:39.047266 15904 leader_election.cc:341] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269): Not found: Tablet not found: 184b94869cdc4b4a9af3614448a77f75
I20260811 02:49:39.047369 15904 leader_election.cc:302] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1d236ed396945fea79e9f4c17bdb879; no voters: 1601bebbded246e3874fc777ac130b88, 4be7e38240de4b3b9c2b61aa129f4a6f
I20260811 02:49:39.047683 16576 raft_consensus.cc:2726] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:49:39.440788 16577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:39.440368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:32883)
0811 02:49:39.440540 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:39.440652 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 02:49:39.565588 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:39.565132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:58615)
0811 02:49:39.565406 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:39.565474 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":98,"threads_started":1}
I20260811 02:49:39.758198 16588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:39.757730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:57839)
0811 02:49:39.758051 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:39.758109 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":78,"threads_started":1}
I20260811 02:49:40.442271 16590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:40.441709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:44469)
0811 02:49:40.442013 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:40.442140 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":106,"threads_started":1}
I20260811 02:49:40.566782 16591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:40.566347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:52691)
0811 02:49:40.566632 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:40.566688 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":100,"threads_started":1}
I20260811 02:49:40.759781 16592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:40.759239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:39539)
0811 02:49:40.759588 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:40.759655 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":144,"threads_started":1}
I20260811 02:49:41.113529 16594 raft_consensus.cc:482] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:49:41.113668 16594 raft_consensus.cc:504] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:41.114140 16594 leader_election.cc:288] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397), 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269)
W20260811 02:49:41.114988 15906 leader_election.cc:341] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397): Not found: Tablet not found: 184b94869cdc4b4a9af3614448a77f75
W20260811 02:49:41.115171 15904 leader_election.cc:341] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269): Not found: Tablet not found: 184b94869cdc4b4a9af3614448a77f75
I20260811 02:49:41.115249 15904 leader_election.cc:302] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1d236ed396945fea79e9f4c17bdb879; no voters: 1601bebbded246e3874fc777ac130b88, 4be7e38240de4b3b9c2b61aa129f4a6f
I20260811 02:49:41.115448 16594 raft_consensus.cc:2726] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:49:41.443531 16595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:41.443095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:43327)
0811 02:49:41.443368 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:41.443428 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":99,"threads_started":1}
I20260811 02:49:41.567883 16597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:41.567462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:44317)
0811 02:49:41.567730 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:41.567782 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":95,"threads_started":1}
I20260811 02:49:41.761023 16598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:41.760581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:40103)
0811 02:49:41.760868 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:41.760931 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":82,"threads_started":1}
I20260811 02:49:42.444970 16599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:42.444493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:56977)
0811 02:49:42.444817 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:42.444870 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":75,"threads_started":1}
I20260811 02:49:42.569201 16600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:42.568766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:33883)
0811 02:49:42.569044 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:42.569105 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":84,"threads_started":1}
I20260811 02:49:42.762374 16601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:42.761898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:49573)
0811 02:49:42.762210 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:42.762274 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":137,"threads_started":1}
I20260811 02:49:43.031798 16604 raft_consensus.cc:482] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:49:43.031977 16604 raft_consensus.cc:504] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:43.032467 16604 leader_election.cc:288] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397), 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269)
W20260811 02:49:43.033280 15904 leader_election.cc:341] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269): Not found: Tablet not found: 184b94869cdc4b4a9af3614448a77f75
W20260811 02:49:43.033392 15906 leader_election.cc:341] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397): Not found: Tablet not found: 184b94869cdc4b4a9af3614448a77f75
I20260811 02:49:43.033499 15906 leader_election.cc:302] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1d236ed396945fea79e9f4c17bdb879; no voters: 1601bebbded246e3874fc777ac130b88, 4be7e38240de4b3b9c2b61aa129f4a6f
I20260811 02:49:43.033654 16604 raft_consensus.cc:2726] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:49:43.446362 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:43.445853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:42461)
0811 02:49:43.446145 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:43.446208 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":85,"threads_started":1}
I20260811 02:49:43.570564 16609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:43.570078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:37347)
0811 02:49:43.570397 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:43.570463 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":97,"threads_started":1}
I20260811 02:49:43.764068 16610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:43.763448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:53421)
0811 02:49:43.763857 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:43.763933 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":85,"threads_started":1}
I20260811 02:49:44.447670 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:44.447243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:38013)
0811 02:49:44.447515 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:44.447572 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
I20260811 02:49:44.571902 16614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:44.571446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:60923)
0811 02:49:44.571745 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:44.571804 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":74,"threads_started":1}
I20260811 02:49:44.765420 16619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:44.764882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:47021)
0811 02:49:44.765259 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:44.765320 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":103,"threads_started":1}
I20260811 02:49:45.274441 16621 raft_consensus.cc:482] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:49:45.274592 16621 raft_consensus.cc:504] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:45.275008 16621 leader_election.cc:288] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397), 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269)
W20260811 02:49:45.275822 15904 leader_election.cc:341] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269): Not found: Tablet not found: 184b94869cdc4b4a9af3614448a77f75
W20260811 02:49:45.275944 15906 leader_election.cc:341] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397): Not found: Tablet not found: 184b94869cdc4b4a9af3614448a77f75
I20260811 02:49:45.276042 15906 leader_election.cc:302] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1d236ed396945fea79e9f4c17bdb879; no voters: 1601bebbded246e3874fc777ac130b88, 4be7e38240de4b3b9c2b61aa129f4a6f
I20260811 02:49:45.276197 16621 raft_consensus.cc:2726] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:49:45.449028 16622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:45.448573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:56515)
0811 02:49:45.448853 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:45.448919 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":85,"threads_started":1}
I20260811 02:49:45.573310 16623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:45.572841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:52193)
0811 02:49:45.573146 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:45.573210 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":88,"threads_started":1}
I20260811 02:49:45.766789 16624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:45.766321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:43535)
0811 02:49:45.766623 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:45.766687 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":98,"threads_started":1}
I20260811 02:49:46.450268 16626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:46.449858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:45767)
0811 02:49:46.450116 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:46.450171 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":80,"threads_started":1}
I20260811 02:49:46.574648 16628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:46.574191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:54063)
0811 02:49:46.574485 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:46.574547 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":101,"threads_started":1}
I20260811 02:49:46.768129 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:46.767655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:48739)
0811 02:49:46.767958 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:46.768024 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":86,"threads_started":1}
I20260811 02:49:47.451772 16632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:47.451322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:52891)
0811 02:49:47.451613 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:47.451673 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":88,"threads_started":1}
I20260811 02:49:47.575842 16633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:47.575428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:47105)
0811 02:49:47.575689 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:47.575747 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":78,"threads_started":1}
I20260811 02:49:47.769512 16636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:47.769034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:47209)
0811 02:49:47.769336 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:47.769409 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":82,"threads_started":1}
I20260811 02:49:48.415549 16637 raft_consensus.cc:482] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:49:48.415692 16637 raft_consensus.cc:504] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1601bebbded246e3874fc777ac130b88" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41397 } } peers { permanent_uuid: "4be7e38240de4b3b9c2b61aa129f4a6f" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43269 } } peers { permanent_uuid: "d1d236ed396945fea79e9f4c17bdb879" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42999 } }
I20260811 02:49:48.416087 16637 leader_election.cc:288] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397), 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269)
W20260811 02:49:48.417143 15904 leader_election.cc:341] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4be7e38240de4b3b9c2b61aa129f4a6f (127.21.236.131:43269): Not found: Tablet not found: 184b94869cdc4b4a9af3614448a77f75
W20260811 02:49:48.417292 15906 leader_election.cc:341] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1601bebbded246e3874fc777ac130b88 (127.21.236.130:41397): Not found: Tablet not found: 184b94869cdc4b4a9af3614448a77f75
I20260811 02:49:48.417443 15906 leader_election.cc:302] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1d236ed396945fea79e9f4c17bdb879; no voters: 1601bebbded246e3874fc777ac130b88, 4be7e38240de4b3b9c2b61aa129f4a6f
I20260811 02:49:48.417650 16637 raft_consensus.cc:2726] T 184b94869cdc4b4a9af3614448a77f75 P d1d236ed396945fea79e9f4c17bdb879 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:49:48.453183 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:48.452701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:39237)
0811 02:49:48.453007 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:48.453075 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":83,"threads_started":1}
I20260811 02:49:48.577178 16640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:48.576719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:47989)
0811 02:49:48.576992 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:48.577070 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":77,"threads_started":1}
I20260811 02:49:48.770963 16641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:48.770455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:37391)
0811 02:49:48.770774 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:48.770840 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":101,"threads_started":1}
I20260811 02:49:49.454501 16642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:49.454024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:39225)
0811 02:49:49.454318 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:49.454387 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":80,"threads_started":1}
I20260811 02:49:49.578449 16644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:49.577995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:39935)
0811 02:49:49.578292 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:49.578350 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":96,"threads_started":1}
I20260811 02:49:49.772362 16645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:49.771858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:59861)
0811 02:49:49.772187 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:49.772257 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":104,"threads_started":1}
I20260811 02:49:50.455931 16646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:50.455453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:52991)
0811 02:49:50.455759 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:50.455818 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":104,"threads_started":1}
I20260811 02:49:50.579793 16647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:50.579332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:58381)
0811 02:49:50.579637 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:50.579696 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":129,"threads_started":1}
I20260811 02:49:50.773761 16649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:50.773247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:54745)
0811 02:49:50.773587 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:50.773650 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":100,"threads_started":1}
I20260811 02:49:51.457196 16651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:51.456754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:57805)
0811 02:49:51.457035 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:51.457095 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":103,"threads_started":1}
I20260811 02:49:51.580947 16653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:51.580492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:40517)
0811 02:49:51.580782 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:51.580839 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":103,"threads_started":1}
I20260811 02:49:51.775187 16655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:51.774595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:38689)
0811 02:49:51.774936 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:51.775009 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":107,"threads_started":1}
I20260811 02:49:52.458758 16657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:52.458258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.129:46327)
0811 02:49:52.458568 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:52.458628 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":121,"threads_started":1}
I20260811 02:49:52.582461 16658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:52.582024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.130:39045)
0811 02:49:52.582295 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:52.582360 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":83,"threads_started":1}
I20260811 02:49:52.755154 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 15800
I20260811 02:49:52.768287 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 16110
I20260811 02:49:52.779134 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:52.778541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:43417 (local address 127.21.236.131:43547)
0811 02:49:52.778887 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:52.778968 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:43417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":89,"threads_started":1}
I20260811 02:49:52.785018 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 16354
2026-08-11T02:49:52Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19750 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:49:52 dist-test-slave-n326 krb5kdc[16663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:49:52 dist-test-slave-n326 krb5kdc[16663](info): set up 2 sockets
Aug 11 02:49:52 dist-test-slave-n326 krb5kdc[16663](info): commencing operation
krb5kdc: starting...
W20260811 02:49:54.144932 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:49:54 dist-test-slave-n326 krb5kdc[16663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:49:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:49:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:49:54.279620 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:38945
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:38945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:54.401758 16685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:54.402084 16685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:54.402168 16685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:54.406492 16685 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 02:49:54.406665 16685 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 02:49:54.406746 16685 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 02:49:54.406785 16685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:49:54.406827 16685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:54.406863 16685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:54.406896 16685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:49:54.406929 16685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:49:54.412664 16685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:38945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:38945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:54.413836 16685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:54.414909 16685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:54.424590 16695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:54.424602 16705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:54.424602 16701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:54.424815 16685 server_base.cc:1034] running on GCE node
I20260811 02:49:54.425583 16685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:54.426716 16685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:54.427920 16685 hybrid_clock.cc:648] HybridClock initialized: now 1786416594427918 us; error 57 us; skew 500 ppm
Aug 11 02:49:54 dist-test-slave-n326 krb5kdc[16663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416594, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:54.430850 16685 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:49:54.431927 16685 webserver.cc:460] Webserver started at http://127.21.236.190:40279/ using document root <none> and password file <none>
I20260811 02:49:54.432535 16685 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:54.432628 16685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:54.432874 16685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:49:54.435053 16685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "7f45b41402494f5f863b14aed6b222c6"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "e8e9ee6e888614d5b7c1464e2822b27a"
server_key_iv: "7e33ed4d74de0b945a016147d8ef656c"
server_key_version: "encryptionkey@0"
I20260811 02:49:54.435724 16685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "7f45b41402494f5f863b14aed6b222c6"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "e8e9ee6e888614d5b7c1464e2822b27a"
server_key_iv: "7e33ed4d74de0b945a016147d8ef656c"
server_key_version: "encryptionkey@0"
I20260811 02:49:54.439641 16685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:49:54.442135 16712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:54.443486 16685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:49:54.443632 16685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "7f45b41402494f5f863b14aed6b222c6"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "e8e9ee6e888614d5b7c1464e2822b27a"
server_key_iv: "7e33ed4d74de0b945a016147d8ef656c"
server_key_version: "encryptionkey@0"
I20260811 02:49:54.443745 16685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:54.480041 16685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:54.483419 16685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:54.483646 16685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:54.491398 16685 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:38945
I20260811 02:49:54.491677 16789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:38945 every 8 connection(s)
I20260811 02:49:54.492492 16685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:49:54.495412 16790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:54.497191 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 16685
I20260811 02:49:54.497331 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:49:54.497632 22450 external_mini_cluster.cc:1428] Setting key c2c3c444a2ac3eff9deb6c6402089850
I20260811 02:49:54.501883 16790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Bootstrap starting.
I20260811 02:49:54.504225 16790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:54.505070 16790 log.cc:824] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:49:54 dist-test-slave-n326 krb5kdc[16663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:54.507556 16790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: No bootstrap required, opened a new log
I20260811 02:49:54.511831 16790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } }
I20260811 02:49:54.512054 16790 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:54.512113 16790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f45b41402494f5f863b14aed6b222c6, State: Initialized, Role: FOLLOWER
I20260811 02:49:54.512277 16793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:54.499140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42046 (local address 127.21.236.190:38945)
0811 02:49:54.499575 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:54.499590 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:54.499633 (+ 43us) server_negotiation.cc:407] Connection header received
0811 02:49:54.500289 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:54.500306 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:54.500637 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:54.500980 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:54.501831 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:54.502874 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:54.503815 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:54.504337 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:54.504521 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:54.507540 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:54.507545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:54.507558 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:54.507586 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:54.509290 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:54.509832 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:54.509837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:54.509843 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:54.509953 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:54.510233 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:54.510235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:54.510236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:54.510624 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:54.510788 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:54.511098 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 02:49:54.511433 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":141,"threads_started":1}
I20260811 02:49:54.512585 16790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } }
I20260811 02:49:54.512740 16790 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:49:54.512825 16790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:49:54.512925 16790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:54.514040 16790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } }
I20260811 02:49:54.514400 16790 leader_election.cc:302] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f45b41402494f5f863b14aed6b222c6; no voters:
I20260811 02:49:54.514796 16790 leader_election.cc:288] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:49:54.514845 16802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:49:54.515100 16802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 1 LEADER]: Becoming Leader. State: Replica: 7f45b41402494f5f863b14aed6b222c6, State: Running, Role: LEADER
I20260811 02:49:54.515425 16802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } }
I20260811 02:49:54.516134 16790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:49:54.517104 16804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f45b41402494f5f863b14aed6b222c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } } }
I20260811 02:49:54.517247 16804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:54.517691 16818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:49:54.517643 16805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f45b41402494f5f863b14aed6b222c6. Latest consensus state: current_term: 1 leader_uuid: "7f45b41402494f5f863b14aed6b222c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } } }
I20260811 02:49:54.518361 16805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:49:54.520778 16818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:49:54.526059 16818 catalog_manager.cc:1347] Generated new cluster ID: e25854f8154d4b46a59b0ef7ad1fcb08
I20260811 02:49:54.526131 16818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:49:54.539017 16818 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:49:54.540048 16818 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:49:54.552418 16818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Generated new TSK 0
I20260811 02:49:54.553191 16818 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:49:54.592149 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:38945
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:54.705856 16840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:54.706171 16840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:54.706249 16840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:54.710301 16840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:54.710395 16840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:54.710484 16840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:49:54.715859 16840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:38945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:54.717090 16840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:54.718201 16840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:54.728344 16851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:54.728364 16849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:54.728359 16848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:54.728683 16840 server_base.cc:1034] running on GCE node
I20260811 02:49:54.729185 16840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:54.729872 16840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:54.731107 16840 hybrid_clock.cc:648] HybridClock initialized: now 1786416594731037 us; error 45 us; skew 500 ppm
Aug 11 02:49:54 dist-test-slave-n326 krb5kdc[16663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416594, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:54.733991 16840 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:49:54.734941 16840 webserver.cc:460] Webserver started at http://127.21.236.129:38661/ using document root <none> and password file <none>
I20260811 02:49:54.735690 16840 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:54.735786 16840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:54.736007 16840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:49:54.738029 16840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "2c7b922456034f79a691e07b8b211112"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "04c2fbee3d0bc8204e20e5c8cbc4aa76"
server_key_iv: "ce6a32777455c381915803f62798b317"
server_key_version: "encryptionkey@0"
I20260811 02:49:54.738554 16840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "2c7b922456034f79a691e07b8b211112"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "04c2fbee3d0bc8204e20e5c8cbc4aa76"
server_key_iv: "ce6a32777455c381915803f62798b317"
server_key_version: "encryptionkey@0"
I20260811 02:49:54.742702 16840 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:49:54.745325 16860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:54.746917 16840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:49:54.747059 16840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "2c7b922456034f79a691e07b8b211112"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "04c2fbee3d0bc8204e20e5c8cbc4aa76"
server_key_iv: "ce6a32777455c381915803f62798b317"
server_key_version: "encryptionkey@0"
I20260811 02:49:54.747226 16840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:54.761929 16840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:54.765604 16840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:54.765834 16840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:54.766499 16840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:54.767488 16840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:54.767566 16840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:54.767660 16840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:54.767712 16840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:54.777894 16840 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:33407
I20260811 02:49:54.777920 17073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:33407 every 8 connection(s)
I20260811 02:49:54.778852 16840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:49:54.779567 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 16840
I20260811 02:49:54.779726 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:49:54.780187 22450 external_mini_cluster.cc:1428] Setting key 2ee8d1c41721e20a640acfe2e1ee805c
Aug 11 02:49:54 dist-test-slave-n326 krb5kdc[16663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416594, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
I20260811 02:49:54.795528 16793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:54.781182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:58339 (local address 127.21.236.190:38945)
0811 02:49:54.781337 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:54.781342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:54.782802 (+ 1460us) server_negotiation.cc:407] Connection header received
0811 02:49:54.784332 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:54.784334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:54.784385 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:54.784535 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:54.785758 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:54.786634 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:54.788230 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:54.788697 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:54.788830 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:54.791848 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:54.791852 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:54.791855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:54.791884 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:54.793326 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:54.793853 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:54.793856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:54.793857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:54.793934 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:54.794271 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:54.794274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:54.794276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:54.794488 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:54.794569 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:54.795218 (+ 649us) server_negotiation.cc:299] Negotiation successful
0811 02:49:54.795339 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:49:54.796525 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:54.781717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:58339)
0811 02:49:54.782556 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:54.782639 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:54.784051 (+ 1412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:54.784719 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:54.784726 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:54.785107 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:54.785603 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:54.785617 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:54.786796 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:54.786799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:54.788081 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:54.788094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:54.788859 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:54.788862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:54.789024 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:54.789761 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:54.789780 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:54.791657 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:54.793438 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:54.793444 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:54.793459 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:54.793753 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:54.794026 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:54.794029 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:54.794031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:54.794179 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:54.794583 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:54.794588 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:54.794844 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 02:49:54.795119 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 02:49:54.795535 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":119,"threads_started":1}
I20260811 02:49:54.798151 17074 heartbeater.cc:347] Connected to a master server at 127.21.236.190:38945
I20260811 02:49:54.798441 17074 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:54.799048 17074 heartbeater.cc:511] Master 127.21.236.190:38945 requested a full tablet report, sending...
I20260811 02:49:54.800822 16742 ts_manager.cc:194] Registered new tserver with Master: 2c7b922456034f79a691e07b8b211112 (127.21.236.129:33407)
I20260811 02:49:54.802089 16742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:58339
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:49:54.813910 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:38945
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:54.929070 17082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:54.929340 17082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:54.929385 17082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:54.933568 17082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:54.933632 17082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:54.933719 17082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:49:54.939105 17082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:38945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:54.940217 17082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:54.941319 17082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:54.951416 17096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:54.951418 17092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:54.951567 17082 server_base.cc:1034] running on GCE node
W20260811 02:49:54.951735 17098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:54.952231 17082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:54.952903 17082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:54.954064 17082 hybrid_clock.cc:648] HybridClock initialized: now 1786416594954033 us; error 43 us; skew 500 ppm
Aug 11 02:49:54 dist-test-slave-n326 krb5kdc[16663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416594, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:54.956913 17082 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:49:54.957782 17082 webserver.cc:460] Webserver started at http://127.21.236.130:36341/ using document root <none> and password file <none>
I20260811 02:49:54.958313 17082 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:54.958375 17082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:54.958554 17082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:49:54.960665 17082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "438c5c243fee434e90ba4a65974d74a2"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "69bbaa37d79619a114260b25bfffcd8c"
server_key_iv: "2f63f320da2511c9d78e11de645ce0d9"
server_key_version: "encryptionkey@0"
I20260811 02:49:54.961246 17082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "438c5c243fee434e90ba4a65974d74a2"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "69bbaa37d79619a114260b25bfffcd8c"
server_key_iv: "2f63f320da2511c9d78e11de645ce0d9"
server_key_version: "encryptionkey@0"
I20260811 02:49:54.964998 17082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:49:54.967312 17109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:54.968514 17082 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:49:54.968622 17082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "438c5c243fee434e90ba4a65974d74a2"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "69bbaa37d79619a114260b25bfffcd8c"
server_key_iv: "2f63f320da2511c9d78e11de645ce0d9"
server_key_version: "encryptionkey@0"
I20260811 02:49:54.968708 17082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:54.989873 17082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:54.993005 17082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:54.993166 17082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:54.993729 17082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:54.994541 17082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:54.994601 17082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:54.994642 17082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:54.994656 17082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:55.004021 17082 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:36071
I20260811 02:49:55.004088 17319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:36071 every 8 connection(s)
I20260811 02:49:55.005048 17082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:49:55.011817 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 17082
I20260811 02:49:55.011952 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:49:55.012228 22450 external_mini_cluster.cc:1428] Setting key 4391801dfdbc338b3e0c210f95d5e7a6
Aug 11 02:49:55 dist-test-slave-n326 krb5kdc[16663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416594, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:55.020731 16793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.007135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:41831 (local address 127.21.236.190:38945)
0811 02:49:55.007289 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:55.007295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:55.008486 (+ 1191us) server_negotiation.cc:407] Connection header received
0811 02:49:55.009356 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:55.009359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:55.009406 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:55.009515 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:55.010732 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.011287 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.012582 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.012892 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.013005 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.016031 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:55.016038 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:55.016044 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:55.016098 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:55.018093 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:55.018802 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:55.018808 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:55.018813 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:55.018915 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:55.019328 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:55.019335 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:55.019340 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:55.019645 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:55.019761 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:55.020303 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 02:49:55.020463 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:49:55.021430 17324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.007614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:41831)
0811 02:49:55.008337 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:55.008381 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:55.009173 (+ 792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:55.009692 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:55.009698 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:55.010070 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:55.010595 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:55.010609 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.011435 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.011438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:55.012453 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:55.012463 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.013061 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.013065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:55.013254 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.013908 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:55.013924 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:55.015814 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:55.018241 (+ 2427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:55.018246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:55.018271 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:55.018685 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:55.019013 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:55.019015 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:55.019017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:55.019221 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:55.019774 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:55.019779 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:55.020022 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 02:49:55.020189 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 02:49:55.020455 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":123,"threads_started":1}
I20260811 02:49:55.022603 17320 heartbeater.cc:347] Connected to a master server at 127.21.236.190:38945
I20260811 02:49:55.022861 17320 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:55.023443 17320 heartbeater.cc:511] Master 127.21.236.190:38945 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
I20260811 02:49:55.024503 16742 ts_manager.cc:194] Registered new tserver with Master: 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071)
I20260811 02:49:55.025066 16742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:41831
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:49:55.044523 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:38945
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:49:55.160230 17327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:49:55.160498 17327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:49:55.160564 17327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:49:55.164597 17327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:49:55.164664 17327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:49:55.164745 17327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:49:55.170104 17327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:38945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:49:55.171305 17327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:49:55.172355 17327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:49:55.182598 17342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:55.182663 17345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:49:55.182680 17338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:49:55.183001 17327 server_base.cc:1034] running on GCE node
I20260811 02:49:55.183462 17327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:49:55.184182 17327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:49:55.185369 17327 hybrid_clock.cc:648] HybridClock initialized: now 1786416595185344 us; error 33 us; skew 500 ppm
Aug 11 02:49:55 dist-test-slave-n326 krb5kdc[16663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416595, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:49:55.188520 17327 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:49:55.189484 17327 webserver.cc:460] Webserver started at http://127.21.236.131:36075/ using document root <none> and password file <none>
I20260811 02:49:55.190135 17327 fs_manager.cc:359] Metadata directory not provided
I20260811 02:49:55.190237 17327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:49:55.190495 17327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:49:55.192833 17327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "f9549ad6e1dc4d5aa87b832ca045a68b"
format_stamp: "Formatted at 2026-08-11 02:49:55 on dist-test-slave-n326"
server_key: "8ece1e472dacc1cac744537d8143eb52"
server_key_iv: "5efee4e16c6d5b35acd80ec81a3c9815"
server_key_version: "encryptionkey@0"
I20260811 02:49:55.193468 17327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "f9549ad6e1dc4d5aa87b832ca045a68b"
format_stamp: "Formatted at 2026-08-11 02:49:55 on dist-test-slave-n326"
server_key: "8ece1e472dacc1cac744537d8143eb52"
server_key_iv: "5efee4e16c6d5b35acd80ec81a3c9815"
server_key_version: "encryptionkey@0"
I20260811 02:49:55.197608 17327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:49:55.200485 17356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:55.201902 17327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:49:55.202068 17327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "f9549ad6e1dc4d5aa87b832ca045a68b"
format_stamp: "Formatted at 2026-08-11 02:49:55 on dist-test-slave-n326"
server_key: "8ece1e472dacc1cac744537d8143eb52"
server_key_iv: "5efee4e16c6d5b35acd80ec81a3c9815"
server_key_version: "encryptionkey@0"
I20260811 02:49:55.202199 17327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:49:55.225724 17327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:49:55.229466 17327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:49:55.229727 17327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:49:55.230412 17327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:49:55.231458 17327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:49:55.231531 17327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:55.231613 17327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:49:55.231665 17327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:49:55.241613 17327 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:41589
I20260811 02:49:55.241617 17568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:41589 every 8 connection(s)
I20260811 02:49:55.242615 17327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:49:55.251835 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 17327
I20260811 02:49:55.251988 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:49:55.252314 22450 external_mini_cluster.cc:1428] Setting key a4e4346d0786ebe0ed6e7957ab69c178
Aug 11 02:49:55 dist-test-slave-n326 krb5kdc[16663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416595, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:49:55.257570 16793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.244599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:33113 (local address 127.21.236.190:38945)
0811 02:49:55.244762 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:55.244768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:55.246003 (+ 1235us) server_negotiation.cc:407] Connection header received
0811 02:49:55.247425 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:55.247428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:55.247506 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:55.247587 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:55.248848 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.249422 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.250553 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.250788 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.250871 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.253752 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:55.253757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:55.253760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:55.253790 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:55.255431 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:55.255975 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:55.255979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:55.255981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:55.256074 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:55.256430 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:55.256434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:55.256436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:55.256624 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:55.256729 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:55.257266 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 02:49:55.257389 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:49:55.258554 17572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.244982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:33113)
0811 02:49:55.245821 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:55.245878 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:55.247171 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:55.247780 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:55.247787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:55.248191 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:49:55.248713 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:55.248727 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.249541 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.249543 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:55.250431 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:55.250440 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.250942 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.250946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:55.251152 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.251778 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:49:55.251795 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:49:55.253579 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:49:55.255553 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:55.255559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:55.255574 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:55.255845 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:55.256184 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:49:55.256187 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:49:55.256189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:49:55.256333 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:49:55.256730 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:49:55.256738 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:49:55.256984 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 02:49:55.257222 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 02:49:55.257550 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":121,"threads_started":1}
I20260811 02:49:55.259837 17569 heartbeater.cc:347] Connected to a master server at 127.21.236.190:38945
I20260811 02:49:55.260089 17569 heartbeater.cc:464] Registering TS with master...
I20260811 02:49:55.260568 17569 heartbeater.cc:511] Master 127.21.236.190:38945 requested a full tablet report, sending...
I20260811 02:49:55.261626 16742 ts_manager.cc:194] Registered new tserver with Master: f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589)
I20260811 02:49:55.262207 16742 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:33113
I20260811 02:49:55.266469 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:49:55.275995 16793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.268666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42060 (local address 127.21.236.190:38945)
0811 02:49:55.268819 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:55.268824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:55.269152 (+ 328us) server_negotiation.cc:407] Connection header received
0811 02:49:55.269243 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:55.269245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:55.269287 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:55.269357 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:49:55.270086 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.270589 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.271459 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.271663 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.271738 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.273158 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:49:55.273161 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:49:55.273163 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:49:55.273185 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:49:55.274429 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:55.274810 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:55.274812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:55.274813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:55.274886 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:49:55.275182 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:49:55.275184 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:49:55.275185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:49:55.275386 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:49:55.275492 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:55.275701 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 02:49:55.275795 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:49:55.278764 16742 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:49:55.281813 16742 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:49:55.296322 17595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.291257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48030 (local address 127.21.236.129:33407)
0811 02:49:55.291725 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:55.291743 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:55.291766 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:49:55.291846 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:55.291851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:55.292072 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:55.292265 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:55.292568 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.293322 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.294859 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.295616 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.295753 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.295894 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:55.296055 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 02:49:55.296161 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":105,"threads_started":1}
I20260811 02:49:55.296315 17589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.290376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:33407 (local address 127.0.0.1:48030)
0811 02:49:55.290643 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:55.290704 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:55.290847 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:55.292262 (+ 1415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:55.292268 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:55.292291 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:55.292445 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:55.292451 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.293452 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.293458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:55.294715 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:55.294727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.295742 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.295745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:55.295850 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.295885 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 02:49:55.296023 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 02:49:55.296155 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":26,"spinlock_wait_cycles":7552,"thread_start_us":99,"threads_started":1}
I20260811 02:49:55.296698 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.290298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:41589 (local address 127.0.0.1:52540)
0811 02:49:55.290643 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:55.290704 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:55.290847 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:55.292319 (+ 1472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:55.292323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:55.292342 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:55.292453 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:55.292460 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.294030 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.294033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:55.295211 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:55.295222 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.296284 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.296288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:55.296376 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.296397 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 02:49:55.296451 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 02:49:55.296519 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":75,"spinlock_wait_cycles":2688,"thread_start_us":75,"threads_started":1}
I20260811 02:49:55.296898 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.291043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:36071 (local address 127.0.0.1:50106)
0811 02:49:55.291307 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:55.291321 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:55.291414 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:55.292417 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:55.292420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:55.292436 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:55.292502 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:55.292510 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.293983 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.293986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:55.295175 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:55.295185 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.296466 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.296470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:55.296541 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.296560 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 02:49:55.296627 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 02:49:55.296684 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":5,"thread_start_us":104,"threads_started":1}
I20260811 02:49:55.297068 17596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.291420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50106 (local address 127.21.236.130:36071)
0811 02:49:55.291853 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:55.291864 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:55.291882 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:49:55.291946 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:55.291950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:55.292092 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:55.292237 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:55.292644 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.293827 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.295302 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.296139 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.296279 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.296404 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:55.296770 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 02:49:55.296875 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":134,"threads_started":1}
I20260811 02:49:55.298228 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.291028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52540 (local address 127.21.236.131:41589)
0811 02:49:55.291725 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:55.291742 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:55.291767 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:49:55.291846 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:55.291852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:55.292069 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:55.292612 (+ 543us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:55.292779 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.293855 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.295371 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.296153 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.297164 (+ 1011us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.297951 (+ 787us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:55.298001 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 02:49:55.298090 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":158,"threads_started":1}
I20260811 02:49:55.299953 16956 tablet_service.cc:1467] Processing CreateTablet for tablet d8fb9ef78c9848fdbf71b5e10e2b2420 (DEFAULT_TABLE table=test-table [id=715bbf600830435881441f1a50994cb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:49:55.300451 17197 tablet_service.cc:1467] Processing CreateTablet for tablet d8fb9ef78c9848fdbf71b5e10e2b2420 (DEFAULT_TABLE table=test-table [id=715bbf600830435881441f1a50994cb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:49:55.301199 16956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8fb9ef78c9848fdbf71b5e10e2b2420. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:55.301442 17197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8fb9ef78c9848fdbf71b5e10e2b2420. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:55.301102 17447 tablet_service.cc:1467] Processing CreateTablet for tablet d8fb9ef78c9848fdbf71b5e10e2b2420 (DEFAULT_TABLE table=test-table [id=715bbf600830435881441f1a50994cb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:49:55.302067 17447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8fb9ef78c9848fdbf71b5e10e2b2420. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:49:55.307977 17597 tablet_bootstrap.cc:492] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2: Bootstrap starting.
I20260811 02:49:55.309342 17599 tablet_bootstrap.cc:492] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b: Bootstrap starting.
I20260811 02:49:55.310577 17597 tablet_bootstrap.cc:654] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:55.311421 17597 log.cc:824] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:55.311436 17598 tablet_bootstrap.cc:492] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: Bootstrap starting.
I20260811 02:49:55.311501 17599 tablet_bootstrap.cc:654] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:55.312311 17599 log.cc:824] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:55.313489 17598 tablet_bootstrap.cc:654] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: Neither blocks nor log segments found. Creating new log.
I20260811 02:49:55.313575 17597 tablet_bootstrap.cc:492] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2: No bootstrap required, opened a new log
I20260811 02:49:55.313827 17597 ts_tablet_manager.cc:1397] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:49:55.314220 17598 log.cc:824] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: Log is configured to *not* fsync() on all Append() calls
I20260811 02:49:55.314379 17599 tablet_bootstrap.cc:492] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b: No bootstrap required, opened a new log
I20260811 02:49:55.314568 17599 ts_tablet_manager.cc:1397] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:49:55.315995 17598 tablet_bootstrap.cc:492] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: No bootstrap required, opened a new log
I20260811 02:49:55.316213 17598 ts_tablet_manager.cc:1397] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:49:55.318055 17597 raft_consensus.cc:348] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } }
I20260811 02:49:55.318361 17597 raft_consensus.cc:374] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:55.318450 17597 raft_consensus.cc:729] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438c5c243fee434e90ba4a65974d74a2, State: Initialized, Role: FOLLOWER
I20260811 02:49:55.318452 17599 raft_consensus.cc:348] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } }
I20260811 02:49:55.318760 17599 raft_consensus.cc:374] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:55.318825 17599 raft_consensus.cc:729] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9549ad6e1dc4d5aa87b832ca045a68b, State: Initialized, Role: FOLLOWER
I20260811 02:49:55.318946 17597 consensus_queue.cc:260] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } }
I20260811 02:49:55.319280 17599 consensus_queue.cc:260] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } }
I20260811 02:49:55.320155 17597 ts_tablet_manager.cc:1428] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:49:55.320319 17599 ts_tablet_manager.cc:1428] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:49:55.320556 17320 heartbeater.cc:503] Master 127.21.236.190:38945 was elected leader, sending a full tablet report...
I20260811 02:49:55.320633 17569 heartbeater.cc:503] Master 127.21.236.190:38945 was elected leader, sending a full tablet report...
I20260811 02:49:55.320327 17598 raft_consensus.cc:348] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } }
I20260811 02:49:55.321728 17598 raft_consensus.cc:374] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:49:55.321833 17598 raft_consensus.cc:729] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7b922456034f79a691e07b8b211112, State: Initialized, Role: FOLLOWER
I20260811 02:49:55.322253 17598 consensus_queue.cc:260] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } }
I20260811 02:49:55.322917 17074 heartbeater.cc:503] Master 127.21.236.190:38945 was elected leader, sending a full tablet report...
I20260811 02:49:55.323273 17598 ts_tablet_manager.cc:1428] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 02:49:55.330717 17608 raft_consensus.cc:482] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:49:55.330878 17608 raft_consensus.cc:504] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } }
I20260811 02:49:55.331921 17608 leader_election.cc:288] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589), 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071)
I20260811 02:49:55.337467 17596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.332302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:45335 (local address 127.21.236.130:36071)
0811 02:49:55.332425 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:55.332430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:55.332667 (+ 237us) server_negotiation.cc:407] Connection header received
0811 02:49:55.332756 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:55.332759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:55.332811 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:55.332892 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:55.333316 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.334107 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.336021 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.336661 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.336787 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.336840 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:55.337260 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 02:49:55.337335 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 02:49:55.337467 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.332189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:34253 (local address 127.21.236.131:41589)
0811 02:49:55.332327 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:55.332332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:55.332632 (+ 300us) server_negotiation.cc:407] Connection header received
0811 02:49:55.332762 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:55.332765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:55.332808 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:55.332892 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:49:55.333316 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.334099 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.336020 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.336661 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.336786 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.336840 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:55.337260 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 02:49:55.337335 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:49:55.337684 17613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.332146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:36071 (local address 127.21.236.129:45335)
0811 02:49:55.332577 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:55.332591 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:55.332669 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:55.332915 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:55.332918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:55.332958 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:55.333201 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:55.333208 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.334220 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.334224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:55.335886 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:55.335898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.336797 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.336800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:55.337021 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.337145 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 02:49:55.337224 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 02:49:55.337408 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":413,"spinlock_wait_cycles":9088,"thread_start_us":110,"threads_started":1}
I20260811 02:49:55.337947 17612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.332125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:41589 (local address 127.21.236.129:34253)
0811 02:49:55.332487 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:55.332503 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:49:55.332666 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:49:55.332915 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:49:55.332918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:49:55.332956 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:49:55.333200 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:55.333208 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.334220 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.334224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:55.335886 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:49:55.335898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.336797 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.336800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:49:55.337023 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.337145 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 02:49:55.337224 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 02:49:55.337408 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":351,"spinlock_wait_cycles":29568,"thread_start_us":117,"threads_started":1}
I20260811 02:49:55.338356 17233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8fb9ef78c9848fdbf71b5e10e2b2420" candidate_uuid: "2c7b922456034f79a691e07b8b211112" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438c5c243fee434e90ba4a65974d74a2" is_pre_election: true
I20260811 02:49:55.338617 17481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8fb9ef78c9848fdbf71b5e10e2b2420" candidate_uuid: "2c7b922456034f79a691e07b8b211112" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" is_pre_election: true
I20260811 02:49:55.338696 17233 raft_consensus.cc:2454] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7b922456034f79a691e07b8b211112 in term 0.
I20260811 02:49:55.338861 17481 raft_consensus.cc:2454] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7b922456034f79a691e07b8b211112 in term 0.
I20260811 02:49:55.339268 16867 leader_election.cc:302] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2c7b922456034f79a691e07b8b211112, 438c5c243fee434e90ba4a65974d74a2, f9549ad6e1dc4d5aa87b832ca045a68b; no voters:
I20260811 02:49:55.339588 17608 raft_consensus.cc:2781] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:49:55.339671 17608 raft_consensus.cc:482] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:49:55.339702 17608 raft_consensus.cc:3037] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:55.340754 17608 raft_consensus.cc:504] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } }
I20260811 02:49:55.341100 17608 leader_election.cc:288] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 1 election: Requested vote from peers f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589), 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071)
I20260811 02:49:55.341466 17481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8fb9ef78c9848fdbf71b5e10e2b2420" candidate_uuid: "2c7b922456034f79a691e07b8b211112" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b"
I20260811 02:49:55.341466 17233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8fb9ef78c9848fdbf71b5e10e2b2420" candidate_uuid: "2c7b922456034f79a691e07b8b211112" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438c5c243fee434e90ba4a65974d74a2"
I20260811 02:49:55.341596 17481 raft_consensus.cc:3037] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:55.341607 17233 raft_consensus.cc:3037] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:49:55.342900 17481 raft_consensus.cc:2454] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7b922456034f79a691e07b8b211112 in term 1.
I20260811 02:49:55.342900 17233 raft_consensus.cc:2454] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7b922456034f79a691e07b8b211112 in term 1.
I20260811 02:49:55.343235 16867 leader_election.cc:302] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c7b922456034f79a691e07b8b211112, 438c5c243fee434e90ba4a65974d74a2; no voters:
I20260811 02:49:55.343461 17608 raft_consensus.cc:2781] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:49:55.343677 17608 raft_consensus.cc:686] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 1 LEADER]: Becoming Leader. State: Replica: 2c7b922456034f79a691e07b8b211112, State: Running, Role: LEADER
I20260811 02:49:55.344002 17608 consensus_queue.cc:237] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } }
I20260811 02:49:55.347124 16742 catalog_manager.cc:5495] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c7b922456034f79a691e07b8b211112 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "2c7b922456034f79a691e07b8b211112" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } health_report { overall_health: UNKNOWN } } }
I20260811 02:49:55.358560 17595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.355209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48044 (local address 127.21.236.129:33407)
0811 02:49:55.355327 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 02:49:55.355331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:49:55.355385 (+ 54us) server_negotiation.cc:407] Connection header received
0811 02:49:55.355521 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:49:55.355523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:49:55.355575 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:49:55.355683 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:49:55.355886 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.356371 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.357185 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:49:55.357389 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:49:55.357463 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:49:55.357726 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:49:55.358129 (+ 403us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:49:55.358220 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 02:49:55.358397 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 02:49:55.358450 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 02:49:55.364425 17481 raft_consensus.cc:1261] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b [term 1 FOLLOWER]: Refusing update from remote peer 2c7b922456034f79a691e07b8b211112: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:49:55.364425 17233 raft_consensus.cc:1261] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2 [term 1 FOLLOWER]: Refusing update from remote peer 2c7b922456034f79a691e07b8b211112: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:49:55.365163 17614 consensus_queue.cc:1035] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:49:55.365401 17608 consensus_queue.cc:1035] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:49:55.372201 17617 mvcc.cc:205] Tried to move back new op lower bound from 7317162374605594624 to 7317162374531817472. Current Snapshot: MvccSnapshot[applied={T|T < 7317162374605594624}]
I20260811 02:49:55.382620 16740 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:49:55.382818 16740 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:49:55.385396 16740 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Sending DeleteTablet for 3 replicas of tablet d8fb9ef78c9848fdbf71b5e10e2b2420
I20260811 02:49:55.386317 17447 tablet_service.cc:1514] Processing DeleteTablet for tablet d8fb9ef78c9848fdbf71b5e10e2b2420 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:49:55 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:52540
I20260811 02:49:55.386300 16956 tablet_service.cc:1514] Processing DeleteTablet for tablet d8fb9ef78c9848fdbf71b5e10e2b2420 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:49:55 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:48030
I20260811 02:49:55.386463 17197 tablet_service.cc:1514] Processing DeleteTablet for tablet d8fb9ef78c9848fdbf71b5e10e2b2420 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:49:55 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:50106
I20260811 02:49:55.386873 17635 tablet_replica.cc:330] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b: stopping tablet replica
I20260811 02:49:55.386894 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 16840
I20260811 02:49:55.400295 17637 tablet_replica.cc:330] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2: stopping tablet replica
I20260811 02:49:55.401027 17637 raft_consensus.cc:2229] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:49:55.401530 17637 raft_consensus.cc:2258] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:49:55.402503 17635 raft_consensus.cc:2229] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:49:55.402911 17635 raft_consensus.cc:2258] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:49:55.403908 17637 ts_tablet_manager.cc:1905] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:49:55.404620 17635 ts_tablet_manager.cc:1905] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:49:55.406569 17637 ts_tablet_manager.cc:1918] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:49:55.406692 17637 log.cc:1196] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/d8fb9ef78c9848fdbf71b5e10e2b2420
I20260811 02:49:55.407111 17637 ts_tablet_manager.cc:1939] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 438c5c243fee434e90ba4a65974d74a2: Deleting consensus metadata
I20260811 02:49:55.408238 17635 ts_tablet_manager.cc:1918] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:49:55.408386 17635 log.cc:1196] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/d8fb9ef78c9848fdbf71b5e10e2b2420
I20260811 02:49:55.408528 16715 catalog_manager.cc:4841] TS 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071): tablet d8fb9ef78c9848fdbf71b5e10e2b2420 (table test-table [id=715bbf600830435881441f1a50994cb3]) successfully deleted
I20260811 02:49:55.408730 17635 ts_tablet_manager.cc:1939] T d8fb9ef78c9848fdbf71b5e10e2b2420 P f9549ad6e1dc4d5aa87b832ca045a68b: Deleting consensus metadata
I20260811 02:49:55.410058 16714 catalog_manager.cc:4841] TS f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589): tablet d8fb9ef78c9848fdbf71b5e10e2b2420 (table test-table [id=715bbf600830435881441f1a50994cb3]) successfully deleted
W20260811 02:49:55.410435 16715 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:33407 (error 108)
I20260811 02:49:55.410732 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 17082
I20260811 02:49:55.417948 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:49:55.417635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:33407 (local address 127.0.0.1:48060)
0811 02:49:55.417780 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 02:49:55.417867 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:33407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 02:49:55.418128 16715 catalog_manager.cc:4584] TS 2c7b922456034f79a691e07b8b211112 (127.21.236.129:33407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8fb9ef78c9848fdbf71b5e10e2b2420: Network error: Client connection negotiation failed: client connection to 127.21.236.129:33407: connect: Connection refused (error 111)
I20260811 02:49:55.423601 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 17327
I20260811 02:49:55.433681 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 16685
I20260811 02:50:15.459684 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:38945
--webserver_interface=127.21.236.190
--webserver_port=40279
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:38945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:50:15.585554 17644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:15.585852 17644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:15.585894 17644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:15.590090 17644 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 02:50:15.590157 17644 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 02:50:15.590174 17644 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 02:50:15.590189 17644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:50:15.590209 17644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:15.590258 17644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:15.590297 17644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:50:15.590320 17644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:50:15.595896 17644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:38945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:38945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=40279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:15.597072 17644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:15.598165 17644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:15.614301 17661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:15.614324 17658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:15.614336 17657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:15.614677 17644 server_base.cc:1034] running on GCE node
I20260811 02:50:15.615512 17644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:15.616863 17644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:15.618078 17644 hybrid_clock.cc:648] HybridClock initialized: now 1786416615618048 us; error 44 us; skew 500 ppm
Aug 11 02:50:15 dist-test-slave-n326 krb5kdc[16663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416615, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:15.627926 17644 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:50:15.629169 17644 webserver.cc:460] Webserver started at http://127.21.236.190:40279/ using document root <none> and password file <none>
I20260811 02:50:15.629871 17644 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:15.630002 17644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:15.634742 17644 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:50:15.637461 17672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:15.639043 17644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 02:50:15.639297 17644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "7f45b41402494f5f863b14aed6b222c6"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "e8e9ee6e888614d5b7c1464e2822b27a"
server_key_iv: "7e33ed4d74de0b945a016147d8ef656c"
server_key_version: "encryptionkey@0"
I20260811 02:50:15.640045 17644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:15.655874 17644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:15.659914 17644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:15.660112 17644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:15.668473 17644 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:38945
I20260811 02:50:15.668821 17758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:38945 every 8 connection(s)
I20260811 02:50:15.669767 17644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:50:15.674571 17761 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:50:15.676842 17761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Bootstrap starting.
I20260811 02:50:15.679422 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 17644
I20260811 02:50:15.680037 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:33407
--local_ip_for_outbound_sockets=127.21.236.129
--tserver_master_addrs=127.21.236.190:38945
--webserver_port=38661
--webserver_interface=127.21.236.129
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:50:15.701061 17761 log.cc:824] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:15.714339 17761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:50:15.714951 17761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Bootstrap complete.
I20260811 02:50:15.720796 17761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } }
I20260811 02:50:15.721297 17761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f45b41402494f5f863b14aed6b222c6, State: Initialized, Role: FOLLOWER
I20260811 02:50:15.721992 17761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } }
I20260811 02:50:15.722170 17761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:50:15.722286 17761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:50:15.722430 17761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 1 FOLLOWER]: Advancing to term 2
I20260811 02:50:15.724798 17761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } }
I20260811 02:50:15.725208 17761 leader_election.cc:302] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f45b41402494f5f863b14aed6b222c6; no voters:
I20260811 02:50:15.725637 17761 leader_election.cc:288] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 02:50:15.725766 17769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 02:50:15.726073 17769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 2 LEADER]: Becoming Leader. State: Replica: 7f45b41402494f5f863b14aed6b222c6, State: Running, Role: LEADER
I20260811 02:50:15.726471 17769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } }
I20260811 02:50:15.727236 17761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:50:15.727814 17769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f45b41402494f5f863b14aed6b222c6. Latest consensus state: current_term: 2 leader_uuid: "7f45b41402494f5f863b14aed6b222c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } } }
I20260811 02:50:15.728744 17769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:50:15.728621 17770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7f45b41402494f5f863b14aed6b222c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } } }
I20260811 02:50:15.729679 17770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:50:15.730074 17786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:50:15.735321 17786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:50:15.736073 17786 catalog_manager.cc:1259] Loaded cluster ID: e25854f8154d4b46a59b0ef7ad1fcb08
I20260811 02:50:15.736194 17786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:50:15.738781 17786 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:50:15.739815 17786 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Loaded TSK: 0
I20260811 02:50:15.755177 17786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Generated new TSK 1
I20260811 02:50:15.755487 17786 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 02:50:15.823885 17764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:15.824169 17764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:15.824213 17764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:15.828483 17764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:15.828663 17764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:15.828778 17764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:50:15.834358 17764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:33407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=38661
--tserver_master_addrs=127.21.236.190:38945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:15.835513 17764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:15.836560 17764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:15.846702 17809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:15.846702 17811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:15.847190 17814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:15.847365 17764 server_base.cc:1034] running on GCE node
I20260811 02:50:15.847774 17764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:15.848523 17764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:15.849761 17764 hybrid_clock.cc:648] HybridClock initialized: now 1786416615849727 us; error 49 us; skew 500 ppm
Aug 11 02:50:15 dist-test-slave-n326 krb5kdc[16663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416615, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:15.852684 17764 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:50:15.853708 17764 webserver.cc:460] Webserver started at http://127.21.236.129:38661/ using document root <none> and password file <none>
I20260811 02:50:15.854302 17764 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:15.854377 17764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:15.858527 17764 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:50:15.860980 17823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:15.862340 17764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:50:15.862443 17764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "2c7b922456034f79a691e07b8b211112"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "04c2fbee3d0bc8204e20e5c8cbc4aa76"
server_key_iv: "ce6a32777455c381915803f62798b317"
server_key_version: "encryptionkey@0"
I20260811 02:50:15.863022 17764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:15.890836 17764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:15.894618 17764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:15.894835 17764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:15.895730 17764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:15.896936 17839 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 02:50:15.901083 17764 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 02:50:15.901161 17764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260811 02:50:15.901252 17764 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 02:50:15.903020 17764 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 02:50:15.903119 17764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260811 02:50:15.903434 17839 tablet_bootstrap.cc:492] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: Bootstrap starting.
I20260811 02:50:15.914501 17764 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:33407
I20260811 02:50:15.914588 18033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:33407 every 8 connection(s)
I20260811 02:50:15.915629 17764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:50:15.920495 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 17764
I20260811 02:50:15.921042 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:36071
--local_ip_for_outbound_sockets=127.21.236.130
--tserver_master_addrs=127.21.236.190:38945
--webserver_port=36341
--webserver_interface=127.21.236.130
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:50:15.923676 17839 log.cc:824] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:50:15 dist-test-slave-n326 krb5kdc[16663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416615, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:50:15.934098 17839 tablet_bootstrap.cc:492] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:50:15.934654 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:15.918186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:38383 (local address 127.21.236.190:38945)
0811 02:50:15.918662 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:15.918682 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:15.918833 (+ 151us) server_negotiation.cc:407] Connection header received
0811 02:50:15.920541 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:15.920558 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:15.920982 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:15.921374 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:15.922948 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:15.924454 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:15.925692 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:15.926175 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:15.926362 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:15.929323 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:15.929351 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:15.929370 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:15.929406 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:15.931350 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:15.931923 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:15.931929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:15.931933 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:15.932056 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:15.932411 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:15.932415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:15.932417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:15.932783 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:15.932934 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:15.933472 (+ 538us) server_negotiation.cc:299] Negotiation successful
0811 02:50:15.933748 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":111,"threads_started":1}
I20260811 02:50:15.934662 17839 tablet_bootstrap.cc:492] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: Bootstrap complete.
I20260811 02:50:15.934665 18041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:15.918023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:38383)
0811 02:50:15.918646 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:15.918705 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:15.920027 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:15.921600 (+ 1573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:15.921609 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:15.922113 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:15.922737 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:15.922758 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:15.924630 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:15.924637 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:15.925552 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:15.925563 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:15.926331 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:15.926336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:15.926539 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:15.927306 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:50:15.927337 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:15.929121 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:15.931488 (+ 2367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:15.931494 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:15.931507 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:15.931794 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:15.932159 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:15.932163 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:15.932165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:15.932311 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:15.932905 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:15.932912 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:15.933168 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 02:50:15.933356 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 02:50:15.933709 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":179,"threads_started":1}
I20260811 02:50:15.936435 17839 ts_tablet_manager.cc:1397] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: Time spent bootstrapping tablet: real 0.033s user 0.017s sys 0.014s
I20260811 02:50:15.937513 18034 heartbeater.cc:347] Connected to a master server at 127.21.236.190:38945
I20260811 02:50:15.937753 18034 heartbeater.cc:464] Registering TS with master...
I20260811 02:50:15.938248 18034 heartbeater.cc:511] Master 127.21.236.190:38945 requested a full tablet report, sending...
I20260811 02:50:15.939472 17839 raft_consensus.cc:348] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } }
I20260811 02:50:15.940007 17839 raft_consensus.cc:729] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7b922456034f79a691e07b8b211112, State: Initialized, Role: FOLLOWER
I20260811 02:50:15.940492 17839 consensus_queue.cc:260] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } }
I20260811 02:50:15.940908 17699 ts_manager.cc:194] Registered new tserver with Master: 2c7b922456034f79a691e07b8b211112 (127.21.236.129:33407)
I20260811 02:50:15.941541 17839 ts_tablet_manager.cc:1428] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 02:50:15.944561 17699 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:38383
I20260811 02:50:15.947256 18034 heartbeater.cc:503] Master 127.21.236.190:38945 was elected leader, sending a full tablet report...
Aug 11 02:50:15 dist-test-slave-n326 krb5kdc[16663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416615, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for kudu/127.21.236.129@KRBTEST.COM
I20260811 02:50:15.955363 18049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:15.943877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:33407 (local address 127.0.0.1:51842)
0811 02:50:15.944489 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:15.944544 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:15.944677 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:15.945970 (+ 1293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:15.945977 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:15.946450 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:15.946601 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:15.946610 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:15.947997 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:15.948000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:15.948645 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:15.948654 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:15.949413 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:15.949417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:15.949533 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:15.950130 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:50:15.950142 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:15.951637 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:15.953568 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:15.953585 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:15.953590 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:15.953818 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:15.954135 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:15.954138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:15.954140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:15.954228 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:15.954686 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:15.954692 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:15.954821 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 02:50:15.954959 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 02:50:15.955178 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":109,"threads_started":1}
I20260811 02:50:15.955395 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:15.943975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51842 (local address 127.21.236.129:33407)
0811 02:50:15.945496 (+ 1521us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:15.945510 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:15.945536 (+ 26us) server_negotiation.cc:407] Connection header received
0811 02:50:15.945593 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:15.945596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:15.945802 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:15.945941 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:15.946837 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:15.947882 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:15.948774 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:15.949279 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:15.949397 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:15.951779 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:15.951789 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:15.951801 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:15.951833 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:15.953439 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:15.953937 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:15.953943 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:15.953947 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:15.954030 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:15.954323 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:15.954327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:15.954329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:15.954567 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:15.954712 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:15.955053 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 02:50:15.955250 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1420,"thread_start_us":98,"threads_started":1}
I20260811 02:50:15.956346 17918 tablet_service.cc:1514] Processing DeleteTablet for tablet d8fb9ef78c9848fdbf71b5e10e2b2420 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:49:55 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:51842
I20260811 02:50:15.957248 18051 tablet_replica.cc:330] stopping tablet replica
I20260811 02:50:15.957557 18051 raft_consensus.cc:2229] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:50:15.957839 18051 raft_consensus.cc:2258] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:50:15.958799 18051 ts_tablet_manager.cc:1905] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:50:15.962350 18051 ts_tablet_manager.cc:1918] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:50:15.962440 18051 log.cc:1196] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal/wals/d8fb9ef78c9848fdbf71b5e10e2b2420
I20260811 02:50:15.962744 18051 ts_tablet_manager.cc:1939] T d8fb9ef78c9848fdbf71b5e10e2b2420 P 2c7b922456034f79a691e07b8b211112: Deleting consensus metadata
I20260811 02:50:15.964545 17675 catalog_manager.cc:4841] TS 2c7b922456034f79a691e07b8b211112 (127.21.236.129:33407): tablet d8fb9ef78c9848fdbf71b5e10e2b2420 (table test-table [id=715bbf600830435881441f1a50994cb3]) successfully deleted
W20260811 02:50:16.050336 18043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:16.050652 18043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:16.050699 18043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:16.055226 18043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:16.055318 18043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:16.055383 18043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:50:16.061156 18043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:36071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=36341
--tserver_master_addrs=127.21.236.190:38945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:16.062340 18043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:16.063457 18043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:16.074958 18057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:16.075106 18043 server_base.cc:1034] running on GCE node
W20260811 02:50:16.074973 18061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:16.074965 18058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:16.075805 18043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:16.076526 18043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:16.077759 18043 hybrid_clock.cc:648] HybridClock initialized: now 1786416616077705 us; error 68 us; skew 500 ppm
Aug 11 02:50:16 dist-test-slave-n326 krb5kdc[16663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416616, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:16.080837 18043 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:50:16.081815 18043 webserver.cc:460] Webserver started at http://127.21.236.130:36341/ using document root <none> and password file <none>
I20260811 02:50:16.082451 18043 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:16.082573 18043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:16.086823 18043 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:50:16.089243 18076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:16.090945 18043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:50:16.091192 18043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "438c5c243fee434e90ba4a65974d74a2"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "69bbaa37d79619a114260b25bfffcd8c"
server_key_iv: "2f63f320da2511c9d78e11de645ce0d9"
server_key_version: "encryptionkey@0"
I20260811 02:50:16.091869 18043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:16.113207 18043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:16.117784 18043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:16.118057 18043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:16.118772 18043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:16.119756 18043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:50:16.119807 18043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:16.119882 18043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:16.119911 18043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:16.134992 18043 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:36071
I20260811 02:50:16.135025 18267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:36071 every 8 connection(s)
I20260811 02:50:16.136250 18043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:50:16.140645 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 18043
I20260811 02:50:16.141180 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:41589
--local_ip_for_outbound_sockets=127.21.236.131
--tserver_master_addrs=127.21.236.190:38945
--webserver_port=36075
--webserver_interface=127.21.236.131
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 02:50:16 dist-test-slave-n326 krb5kdc[16663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416616, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:50:16.155530 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.139090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:49465 (local address 127.21.236.190:38945)
0811 02:50:16.139319 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:16.139323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:16.140616 (+ 1293us) server_negotiation.cc:407] Connection header received
0811 02:50:16.142094 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:16.142097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:16.142157 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:16.142243 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:16.144225 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.145435 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.146835 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.147320 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.147478 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.150679 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:16.150687 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:16.150692 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:16.150731 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:16.152872 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:16.153500 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:16.153505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:16.153506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:16.153613 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:16.153998 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:16.154005 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:16.154008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:16.154323 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:16.154445 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:16.155041 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 02:50:16.155280 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260811 02:50:16.156447 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.139517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:49465)
0811 02:50:16.140411 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:16.140480 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:16.141748 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:16.142479 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:16.142491 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:16.143109 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:16.143812 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:16.143853 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.145685 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.145693 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:16.146686 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:16.146697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.147497 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.147500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:16.147689 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.148492 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:50:16.148511 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:16.150458 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:16.153045 (+ 2587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:16.153052 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:16.153067 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:16.153375 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:16.153724 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:16.153726 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:16.153728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:16.153892 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:16.154468 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:16.154474 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:16.154766 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 02:50:16.154951 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 02:50:16.155361 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":126,"threads_started":1}
I20260811 02:50:16.157998 18270 heartbeater.cc:347] Connected to a master server at 127.21.236.190:38945
I20260811 02:50:16.158315 18270 heartbeater.cc:464] Registering TS with master...
I20260811 02:50:16.158854 18270 heartbeater.cc:511] Master 127.21.236.190:38945 requested a full tablet report, sending...
I20260811 02:50:16.160115 17699 ts_manager.cc:194] Registered new tserver with Master: 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071)
I20260811 02:50:16.160746 17699 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:49465
W20260811 02:50:16.276943 18273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:16.277304 18273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:16.277386 18273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:16.281672 18273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:16.281780 18273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:16.281885 18273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:50:16.287595 18273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:41589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=36075
--tserver_master_addrs=127.21.236.190:38945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:16.288823 18273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:16.289951 18273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:16.299932 18292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:16.299932 18288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:16.299989 18290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:16.300347 18273 server_base.cc:1034] running on GCE node
I20260811 02:50:16.300889 18273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:16.301623 18273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:16.302812 18273 hybrid_clock.cc:648] HybridClock initialized: now 1786416616302788 us; error 35 us; skew 500 ppm
Aug 11 02:50:16 dist-test-slave-n326 krb5kdc[16663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416616, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:16.305949 18273 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:50:16.306905 18273 webserver.cc:460] Webserver started at http://127.21.236.131:36075/ using document root <none> and password file <none>
I20260811 02:50:16.307566 18273 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:16.307670 18273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:16.311789 18273 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:50:16.314045 18301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:16.315465 18273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:50:16.315601 18273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "f9549ad6e1dc4d5aa87b832ca045a68b"
format_stamp: "Formatted at 2026-08-11 02:49:55 on dist-test-slave-n326"
server_key: "8ece1e472dacc1cac744537d8143eb52"
server_key_iv: "5efee4e16c6d5b35acd80ec81a3c9815"
server_key_version: "encryptionkey@0"
I20260811 02:50:16.316179 18273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:16.331904 18273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:16.335469 18273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:16.335700 18273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:16.336346 18273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:16.337239 18273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:50:16.337289 18273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:16.337330 18273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:16.337384 18273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:16.347434 18273 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:41589
I20260811 02:50:16.347471 18500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:41589 every 8 connection(s)
I20260811 02:50:16.348485 18273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:50:16.352041 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 18273
Aug 11 02:50:16 dist-test-slave-n326 krb5kdc[16663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416616, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:50:16.361584 18505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.352769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53294 (local address 127.21.236.190:38945)
0811 02:50:16.353042 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:16.353046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:16.353211 (+ 165us) server_negotiation.cc:407] Connection header received
0811 02:50:16.353364 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:16.353370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:16.353419 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:16.353516 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:16.354296 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.354843 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.355834 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.356182 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.356261 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.357854 (+ 1593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:16.357858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:16.357861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:16.357894 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:16.359697 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:16.360140 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:16.360143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:16.360144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:16.360263 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:16.360618 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:16.360625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:16.360629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:16.360878 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:16.361117 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:16.361286 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 02:50:16.361406 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":191,"thread_start_us":66,"threads_started":1}
I20260811 02:50:16.363461 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.350478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:54843 (local address 127.21.236.190:38945)
0811 02:50:16.350663 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:16.350668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:16.351745 (+ 1077us) server_negotiation.cc:407] Connection header received
0811 02:50:16.352802 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:16.352806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:16.352867 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:16.352965 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:16.354258 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.354810 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.356126 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.356402 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.356517 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.359196 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:16.359201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:16.359203 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:16.359245 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:16.360935 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:16.361605 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:16.361610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:16.361615 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:16.361717 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:16.362228 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:16.362231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:16.362232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:16.362447 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:16.362539 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:16.363166 (+ 627us) server_negotiation.cc:299] Negotiation successful
0811 02:50:16.363292 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":79}
I20260811 02:50:16.364375 18504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.350855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:54843)
0811 02:50:16.351592 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:16.351634 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:16.352540 (+ 906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:16.353259 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:16.353266 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:16.353621 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:16.354110 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:16.354123 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.354961 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.354966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:16.356004 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:16.356015 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.356577 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.356581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:16.356732 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.357245 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:50:16.357261 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:16.358995 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:16.361063 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:16.361070 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:16.361086 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:16.361479 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:16.361828 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:16.361831 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:16.361833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:16.362035 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:16.362589 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:16.362598 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:16.362875 (+ 277us) client_negotiation.cc:769] Sending connection context
0811 02:50:16.363042 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 02:50:16.363527 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":179,"threads_started":1}
I20260811 02:50:16.365468 18502 heartbeater.cc:347] Connected to a master server at 127.21.236.190:38945
I20260811 02:50:16.365751 18502 heartbeater.cc:464] Registering TS with master...
I20260811 02:50:16.366273 18502 heartbeater.cc:511] Master 127.21.236.190:38945 requested a full tablet report, sending...
I20260811 02:50:16.367333 17699 ts_manager.cc:194] Registered new tserver with Master: f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589)
I20260811 02:50:16.367913 17699 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:54843
I20260811 02:50:16.375594 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:50:16.379787 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.376275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.21.236.190:38945)
0811 02:50:16.376436 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:16.376440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:16.376661 (+ 221us) server_negotiation.cc:407] Connection header received
0811 02:50:16.376792 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:16.376794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:16.376927 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:16.377017 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:50:16.377234 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.377752 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.378583 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.378817 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.378939 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.379294 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:50:16.379435 (+ 141us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 02:50:16.379652 (+ 217us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53300: token expired
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 02:50:16.379900 18042 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53300: token expired
W20260811 02:50:16.380236 17578 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.21.236.190:38945)
I20260811 02:50:16.387220 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.380646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53312 (local address 127.21.236.190:38945)
0811 02:50:16.380756 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:16.380759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:16.380771 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:50:16.380825 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:16.380827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:16.380859 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:16.380922 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:16.381565 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.382028 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.382819 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.383014 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.383192 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.384463 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:16.384467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:16.384468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:16.384491 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:16.385789 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:16.386189 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:16.386191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:16.386193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:16.386267 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:16.386499 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:16.386501 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:16.386502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:16.386664 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:16.386783 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:16.386952 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 02:50:16.387053 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 02:50:16.389849 17699 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:50:16.393261 17699 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:50:16.401994 17918 tablet_service.cc:1467] Processing CreateTablet for tablet 0890b3312c454171bae34e9a28c4f38f (DEFAULT_TABLE table=test-table [id=f7d141a800784a24a09ed0601c1c4729]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:50:16.402736 17918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0890b3312c454171bae34e9a28c4f38f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:16.405859 18049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.400647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:41589 (local address 127.0.0.1:52848)
0811 02:50:16.400757 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:16.400774 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:16.400863 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:16.401815 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:16.401819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:16.401841 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:16.401951 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:16.401958 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.403457 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.403460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:16.404456 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:16.404466 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.405362 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.405369 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:16.405465 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.405512 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 02:50:16.405608 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 02:50:16.405694 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 02:50:16.406093 17839 tablet_bootstrap.cc:492] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112: Bootstrap starting.
I20260811 02:50:16.406244 18513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.400856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52848 (local address 127.21.236.131:41589)
0811 02:50:16.401261 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:16.401276 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:16.401304 (+ 28us) server_negotiation.cc:407] Connection header received
0811 02:50:16.401407 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:16.401417 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:16.401660 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:16.401828 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:16.402074 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.403328 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.404591 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.405161 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.405363 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.405522 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:16.405641 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 02:50:16.405867 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":163,"threads_started":1}
I20260811 02:50:16.407004 17839 tablet_bootstrap.cc:654] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:16.407331 18516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.400951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:36071 (local address 127.0.0.1:54288)
0811 02:50:16.401262 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:16.401276 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:16.401376 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:16.402522 (+ 1146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:16.402528 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:16.402561 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:16.402653 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:16.402660 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.404017 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.404021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:16.405128 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:16.405142 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.406852 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.406859 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:16.406983 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.407009 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 02:50:16.407115 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 02:50:16.407200 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":111,"threads_started":1}
I20260811 02:50:16.407435 18520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.401431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54288 (local address 127.21.236.130:36071)
0811 02:50:16.401772 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:16.401784 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:16.401807 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:50:16.401886 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:16.401890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:16.402082 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:16.402320 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:16.402783 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.403887 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.406010 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.406725 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.406829 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.406915 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:16.407164 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 02:50:16.407290 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":156,"threads_started":1}
I20260811 02:50:16.408706 17839 tablet_bootstrap.cc:492] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112: No bootstrap required, opened a new log
I20260811 02:50:16.408802 17839 ts_tablet_manager.cc:1397] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 02:50:16.409257 17839 raft_consensus.cc:348] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:16.409387 17839 raft_consensus.cc:374] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:16.409430 17839 raft_consensus.cc:729] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7b922456034f79a691e07b8b211112, State: Initialized, Role: FOLLOWER
I20260811 02:50:16.409559 17839 consensus_queue.cc:260] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:16.409817 17839 ts_tablet_manager.cc:1428] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 02:50:16.410280 18161 tablet_service.cc:1467] Processing CreateTablet for tablet 0890b3312c454171bae34e9a28c4f38f (DEFAULT_TABLE table=test-table [id=f7d141a800784a24a09ed0601c1c4729]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:50:16.410279 18387 tablet_service.cc:1467] Processing CreateTablet for tablet 0890b3312c454171bae34e9a28c4f38f (DEFAULT_TABLE table=test-table [id=f7d141a800784a24a09ed0601c1c4729]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:50:16.411394 18161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0890b3312c454171bae34e9a28c4f38f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:16.411420 18387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0890b3312c454171bae34e9a28c4f38f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 02:50:16.416909 18036 tablet.cc:2369] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:50:16.419231 18524 tablet_bootstrap.cc:492] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: Bootstrap starting.
I20260811 02:50:16.421878 18524 tablet_bootstrap.cc:654] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:16.422679 18524 log.cc:824] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:16.423416 18523 tablet_bootstrap.cc:492] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b: Bootstrap starting.
I20260811 02:50:16.425084 18524 tablet_bootstrap.cc:492] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: No bootstrap required, opened a new log
I20260811 02:50:16.425295 18524 ts_tablet_manager.cc:1397] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:50:16.426169 18523 tablet_bootstrap.cc:654] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:16.427057 18523 log.cc:824] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:16.429430 18523 tablet_bootstrap.cc:492] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b: No bootstrap required, opened a new log
I20260811 02:50:16.429646 18523 ts_tablet_manager.cc:1397] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:50:16.429584 18524 raft_consensus.cc:348] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:16.429898 18524 raft_consensus.cc:374] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:16.429962 18524 raft_consensus.cc:729] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438c5c243fee434e90ba4a65974d74a2, State: Initialized, Role: FOLLOWER
I20260811 02:50:16.430438 18524 consensus_queue.cc:260] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:16.431173 18270 heartbeater.cc:503] Master 127.21.236.190:38945 was elected leader, sending a full tablet report...
I20260811 02:50:16.431560 18524 ts_tablet_manager.cc:1428] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:50:16.433254 18523 raft_consensus.cc:348] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:16.433557 18523 raft_consensus.cc:374] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:16.433621 18523 raft_consensus.cc:729] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9549ad6e1dc4d5aa87b832ca045a68b, State: Initialized, Role: FOLLOWER
I20260811 02:50:16.434082 18523 consensus_queue.cc:260] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:16.434737 18502 heartbeater.cc:503] Master 127.21.236.190:38945 was elected leader, sending a full tablet report...
I20260811 02:50:16.436000 18523 ts_tablet_manager.cc:1428] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:50:16.524646 18529 raft_consensus.cc:482] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:50:16.524912 18529 raft_consensus.cc:504] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:16.526171 18529 leader_election.cc:288] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071), 2c7b922456034f79a691e07b8b211112 (127.21.236.129:33407)
I20260811 02:50:16.530630 18504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.526374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:36071 (local address 127.21.236.131:55003)
0811 02:50:16.526520 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:16.526537 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:16.526632 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:16.527014 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:16.527020 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:16.527044 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:16.527212 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:16.527219 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.528247 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.528251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:16.529346 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:16.529356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.530176 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.530180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:16.530290 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.530317 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 02:50:16.530399 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 02:50:16.530473 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 02:50:16.530630 18520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.526523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:55003 (local address 127.21.236.130:36071)
0811 02:50:16.526657 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:16.526662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:16.526679 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:50:16.526740 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:16.526743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:16.526797 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:16.526913 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:16.527325 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.528132 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.529472 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.530055 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.530149 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.530191 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:16.530424 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 02:50:16.530482 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:50:16.531112 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.526668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:33407 (local address 127.21.236.131:34035)
0811 02:50:16.527099 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:16.527114 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:16.527224 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:16.527568 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:16.527571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:16.527591 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:16.527710 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:16.527716 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.528750 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.528752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:16.529749 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:16.529760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.530635 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.530642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:16.530764 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.530796 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 02:50:16.530885 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 02:50:16.530936 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":132,"threads_started":1}
I20260811 02:50:16.531160 18534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.526835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:34035 (local address 127.21.236.129:33407)
0811 02:50:16.527200 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:16.527205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:16.527221 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:50:16.527359 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:16.527363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:16.527430 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:16.527524 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:16.527826 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.528649 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.529880 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.530504 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.530619 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.530798 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:16.530894 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 02:50:16.531002 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":80,"threads_started":1}
I20260811 02:50:16.531548 18193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0890b3312c454171bae34e9a28c4f38f" candidate_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438c5c243fee434e90ba4a65974d74a2" is_pre_election: true
I20260811 02:50:16.531824 17950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0890b3312c454171bae34e9a28c4f38f" candidate_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7b922456034f79a691e07b8b211112" is_pre_election: true
I20260811 02:50:16.531909 18193 raft_consensus.cc:2454] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9549ad6e1dc4d5aa87b832ca045a68b in term 0.
I20260811 02:50:16.532122 17950 raft_consensus.cc:2454] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9549ad6e1dc4d5aa87b832ca045a68b in term 0.
I20260811 02:50:16.532418 18304 leader_election.cc:302] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438c5c243fee434e90ba4a65974d74a2, f9549ad6e1dc4d5aa87b832ca045a68b; no voters:
I20260811 02:50:16.532720 18529 raft_consensus.cc:2781] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:50:16.532819 18529 raft_consensus.cc:482] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:50:16.532902 18529 raft_consensus.cc:3037] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:16.533998 18529 raft_consensus.cc:504] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:16.534348 18529 leader_election.cc:288] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [CANDIDATE]: Term 1 election: Requested vote from peers 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071), 2c7b922456034f79a691e07b8b211112 (127.21.236.129:33407)
I20260811 02:50:16.534721 18193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0890b3312c454171bae34e9a28c4f38f" candidate_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438c5c243fee434e90ba4a65974d74a2"
I20260811 02:50:16.534770 17950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0890b3312c454171bae34e9a28c4f38f" candidate_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c7b922456034f79a691e07b8b211112"
I20260811 02:50:16.534861 18193 raft_consensus.cc:3037] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:16.534883 17950 raft_consensus.cc:3037] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:16.536185 18193 raft_consensus.cc:2454] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9549ad6e1dc4d5aa87b832ca045a68b in term 1.
I20260811 02:50:16.536185 17950 raft_consensus.cc:2454] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9549ad6e1dc4d5aa87b832ca045a68b in term 1.
I20260811 02:50:16.536514 18304 leader_election.cc:302] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438c5c243fee434e90ba4a65974d74a2, f9549ad6e1dc4d5aa87b832ca045a68b; no voters:
I20260811 02:50:16.536724 18529 raft_consensus.cc:2781] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:50:16.537026 18529 raft_consensus.cc:686] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [term 1 LEADER]: Becoming Leader. State: Replica: f9549ad6e1dc4d5aa87b832ca045a68b, State: Running, Role: LEADER
I20260811 02:50:16.537384 18529 consensus_queue.cc:237] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:16.540798 17699 catalog_manager.cc:5495] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b reported cstate change: term changed from 0 to 1, leader changed from <none> to f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131). New cstate: current_term: 1 leader_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } health_report { overall_health: UNKNOWN } } }
I20260811 02:50:16.563297 18513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.559605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52850 (local address 127.21.236.131:41589)
0811 02:50:16.559745 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:16.559748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:16.559761 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:50:16.559833 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:16.559836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:16.559888 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:16.559979 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:50:16.560295 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.560822 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.561700 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:16.561930 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:16.562014 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:16.562338 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:50:16.562742 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:50:16.562836 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:16.563005 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 02:50:16.563118 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:50:16.569597 17950 raft_consensus.cc:1261] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Refusing update from remote peer f9549ad6e1dc4d5aa87b832ca045a68b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:50:16.569597 18193 raft_consensus.cc:1261] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2 [term 1 FOLLOWER]: Refusing update from remote peer f9549ad6e1dc4d5aa87b832ca045a68b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:50:16.570421 18535 consensus_queue.cc:1035] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:50:16.570663 18529 consensus_queue.cc:1035] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [LEADER]: Connected to new peer: Peer: permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:50:16.585480 17697 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:50:16.585678 17697 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:50:16.588207 17697 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Sending DeleteTablet for 3 replicas of tablet 0890b3312c454171bae34e9a28c4f38f
I20260811 02:50:16.588910 18387 tablet_service.cc:1514] Processing DeleteTablet for tablet 0890b3312c454171bae34e9a28c4f38f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:50:16 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:52848
I20260811 02:50:16.588959 18161 tablet_service.cc:1514] Processing DeleteTablet for tablet 0890b3312c454171bae34e9a28c4f38f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:50:16 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:54288
I20260811 02:50:16.589105 17918 tablet_service.cc:1514] Processing DeleteTablet for tablet 0890b3312c454171bae34e9a28c4f38f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:50:16 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:51842
I20260811 02:50:16.589212 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 17764
I20260811 02:50:16.598663 18561 tablet_replica.cc:330] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b: stopping tablet replica
I20260811 02:50:16.599160 18561 raft_consensus.cc:2229] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:50:16.599691 18561 raft_consensus.cc:2258] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:50:16.603396 18561 ts_tablet_manager.cc:1905] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:50:16.606185 18561 ts_tablet_manager.cc:1918] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:50:16.606352 18561 log.cc:1196] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/0890b3312c454171bae34e9a28c4f38f
I20260811 02:50:16.606842 18561 ts_tablet_manager.cc:1939] T 0890b3312c454171bae34e9a28c4f38f P f9549ad6e1dc4d5aa87b832ca045a68b: Deleting consensus metadata
I20260811 02:50:16.606910 18560 tablet_replica.cc:330] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: stopping tablet replica
I20260811 02:50:16.607467 18560 raft_consensus.cc:2229] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:50:16.607908 18560 raft_consensus.cc:2258] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:50:16.608356 17674 catalog_manager.cc:4841] TS f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589): tablet 0890b3312c454171bae34e9a28c4f38f (table test-table [id=f7d141a800784a24a09ed0601c1c4729]) successfully deleted
I20260811 02:50:16.609833 18560 ts_tablet_manager.cc:1905] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 02:50:16.611599 17675 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:33407 (error 108)
W20260811 02:50:16.612797 18516 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:50:16.612446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:33407 (local address 127.0.0.1:37410)
0811 02:50:16.612611 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:16.612693 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:33407: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 02:50:16.612866 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 18043
I20260811 02:50:16.612824 18560 ts_tablet_manager.cc:1918] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:50:16.612958 18560 log.cc:1196] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/0890b3312c454171bae34e9a28c4f38f
W20260811 02:50:16.612993 17675 catalog_manager.cc:4584] TS 2c7b922456034f79a691e07b8b211112 (127.21.236.129:33407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0890b3312c454171bae34e9a28c4f38f: Network error: Client connection negotiation failed: client connection to 127.21.236.129:33407: connect: Connection reset by peer (error 104)
I20260811 02:50:16.625264 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 18273
I20260811 02:50:16.625651 18516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:16.625291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:36071 (local address 127.0.0.1:54300)
0811 02:50:16.625474 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:16.625541 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.130:36071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 02:50:16.640607 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 17644
I20260811 02:50:36.651985 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:38945
--webserver_interface=127.21.236.190
--webserver_port=40279
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:38945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:50:36.768743 18564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:36.769078 18564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:36.769176 18564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:36.773641 18564 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 02:50:36.773725 18564 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 02:50:36.773753 18564 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 02:50:36.773790 18564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:50:36.773833 18564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:36.773873 18564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:36.773904 18564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:50:36.773938 18564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:50:36.779232 18564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:38945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:38945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=40279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:36.780354 18564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:36.781430 18564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:36.790125 18586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:36.790117 18590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:36.790133 18584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:36.790469 18564 server_base.cc:1034] running on GCE node
I20260811 02:50:36.791127 18564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:36.792309 18564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:36.793498 18564 hybrid_clock.cc:648] HybridClock initialized: now 1786416636793491 us; error 41 us; skew 500 ppm
Aug 11 02:50:36 dist-test-slave-n326 krb5kdc[16663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416636, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:36.796671 18564 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:50:36.797637 18564 webserver.cc:460] Webserver started at http://127.21.236.190:40279/ using document root <none> and password file <none>
I20260811 02:50:36.798260 18564 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:36.798352 18564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:36.802330 18564 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:50:36.804632 18602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:36.805876 18564 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 02:50:36.806026 18564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "7f45b41402494f5f863b14aed6b222c6"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "e8e9ee6e888614d5b7c1464e2822b27a"
server_key_iv: "7e33ed4d74de0b945a016147d8ef656c"
server_key_version: "encryptionkey@0"
I20260811 02:50:36.806583 18564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:36.818641 18564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:36.821956 18564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:36.822173 18564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:36.829689 18564 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:38945
I20260811 02:50:36.829985 18689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:38945 every 8 connection(s)
I20260811 02:50:36.830890 18564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:50:36.835251 18690 sys_catalog.cc:263] Verifying existing consensus state
I20260811 02:50:36.836931 18690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Bootstrap starting.
I20260811 02:50:36.839923 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 18564
I20260811 02:50:36.840391 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:33407
--local_ip_for_outbound_sockets=127.21.236.129
--tserver_master_addrs=127.21.236.190:38945
--webserver_port=38661
--webserver_interface=127.21.236.129
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:50:36.854884 18690 log.cc:824] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:36.866344 18690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:50:36.866860 18690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Bootstrap complete.
I20260811 02:50:36.871145 18690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } }
I20260811 02:50:36.871568 18690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f45b41402494f5f863b14aed6b222c6, State: Initialized, Role: FOLLOWER
I20260811 02:50:36.872067 18690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } }
I20260811 02:50:36.872213 18690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:50:36.872308 18690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:50:36.872429 18690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 2 FOLLOWER]: Advancing to term 3
I20260811 02:50:36.874361 18690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } }
I20260811 02:50:36.874709 18690 leader_election.cc:302] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f45b41402494f5f863b14aed6b222c6; no voters:
I20260811 02:50:36.875042 18690 leader_election.cc:288] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 02:50:36.875204 18700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 02:50:36.875479 18700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [term 3 LEADER]: Becoming Leader. State: Replica: 7f45b41402494f5f863b14aed6b222c6, State: Running, Role: LEADER
I20260811 02:50:36.875787 18700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } }
I20260811 02:50:36.876308 18690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:50:36.877246 18701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7f45b41402494f5f863b14aed6b222c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } } }
I20260811 02:50:36.877380 18701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:50:36.878000 18702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f45b41402494f5f863b14aed6b222c6. Latest consensus state: current_term: 3 leader_uuid: "7f45b41402494f5f863b14aed6b222c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f45b41402494f5f863b14aed6b222c6" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38945 } } }
I20260811 02:50:36.878182 18702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6 [sys.catalog]: This master's current role is: LEADER
I20260811 02:50:36.878437 18711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:50:36.883677 18711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:50:36.884325 18711 catalog_manager.cc:1259] Loaded cluster ID: e25854f8154d4b46a59b0ef7ad1fcb08
I20260811 02:50:36.884387 18711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:50:36.886791 18711 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:50:36.887552 18711 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Loaded TSK: 0
I20260811 02:50:36.887638 18711 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7f45b41402494f5f863b14aed6b222c6: Loaded TSK: 1
F20260811 02:50:36.888449 18711 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fcdd55e09ad google::LogMessage::Fail() at ??:0
@ 0x7fcdd55e388a google::LogMessage::SendToLog() at ??:0
@ 0x7fcdd55e04de google::LogMessage::Flush() at ??:0
@ 0x7fcdd55e1a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fcdd82a418d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fcdd829cf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fcddff457bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fcddff1f99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fcddff20178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fcddff61df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fcddff8082b std::function<>::operator()() at ??:0
@ 0x7fcddff1fb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fcddff20f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fcddff1d579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fcddff61640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fcdd78323d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fcdd7832c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fcdd78343f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fcdd7823512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fcdd8e676db start_thread at ??:0
@ 0x7fcdd40de61f clone at ??:0
W20260811 02:50:36.987053 18697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:36.987408 18697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:36.987458 18697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:36.991815 18697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:36.991896 18697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:36.992051 18697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:50:36.997785 18697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:33407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=38661
--tserver_master_addrs=127.21.236.190:38945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:36.998981 18697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:37.000140 18697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:37.010849 18765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:37.010787 18762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:37.010819 18763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:37.011780 18697 server_base.cc:1034] running on GCE node
I20260811 02:50:37.012454 18697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:37.013202 18697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:37.014400 18697 hybrid_clock.cc:648] HybridClock initialized: now 1786416637014388 us; error 32 us; skew 500 ppm
Aug 11 02:50:37 dist-test-slave-n326 krb5kdc[16663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416637, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:37.017925 18697 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:50:37.019193 18697 webserver.cc:460] Webserver started at http://127.21.236.129:38661/ using document root <none> and password file <none>
I20260811 02:50:37.019970 18697 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:37.020088 18697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:37.025772 18697 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:50:37.028847 18779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:37.030550 18697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:50:37.030694 18697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "2c7b922456034f79a691e07b8b211112"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "04c2fbee3d0bc8204e20e5c8cbc4aa76"
server_key_iv: "ce6a32777455c381915803f62798b317"
server_key_version: "encryptionkey@0"
I20260811 02:50:37.031553 18697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:37.051446 18697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:37.055711 18697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:37.055945 18697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:37.056842 18697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:37.058351 18790 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 02:50:37.062911 18697 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 02:50:37.063014 18697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 02:50:37.063122 18697 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 02:50:37.065625 18697 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 02:50:37.065696 18697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260811 02:50:37.066020 18790 tablet_bootstrap.cc:492] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112: Bootstrap starting.
I20260811 02:50:37.077919 18697 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:33407
I20260811 02:50:37.078150 18968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:33407 every 8 connection(s)
I20260811 02:50:37.079246 18697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:50:37.086787 18790 log.cc:824] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:37.088943 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 18697
I20260811 02:50:37.089479 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:36071
--local_ip_for_outbound_sockets=127.21.236.130
--tserver_master_addrs=127.21.236.190:38945
--webserver_port=36341
--webserver_interface=127.21.236.130
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:50:37.094835 18790 tablet_bootstrap.cc:492] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 02:50:37.095299 18790 tablet_bootstrap.cc:492] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112: Bootstrap complete.
I20260811 02:50:37.096524 18790 ts_tablet_manager.cc:1397] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112: Time spent bootstrapping tablet: real 0.031s user 0.020s sys 0.009s
I20260811 02:50:37.099205 18790 raft_consensus.cc:348] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:37.099663 18790 raft_consensus.cc:729] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7b922456034f79a691e07b8b211112, State: Initialized, Role: FOLLOWER
I20260811 02:50:37.100152 18790 consensus_queue.cc:260] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:37.101131 18790 ts_tablet_manager.cc:1428] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
W20260811 02:50:37.186664 18972 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:50:37.081781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:53141)
0811 02:50:37.082450 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:37.082509 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:37.083835 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:37.185746 (+101911us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":176,"threads_started":1}
W20260811 02:50:37.187392 18969 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:38945 (0 consecutive failures): Network error: Failed to ping master at 127.21.236.190:38945: Client connection negotiation failed: client connection to 127.21.236.190:38945: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 02:50:37.188158 18972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:37.187887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:44149)
0811 02:50:37.188017 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:37.188101 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 02:50:37.188803 18972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:37.188595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:56635)
0811 02:50:37.188702 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:37.188755 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 02:50:37.188974 18969 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:50:37.241917 18976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:37.242247 18976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:37.242328 18976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:37.246433 18976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:37.246542 18976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:37.246641 18976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:50:37.252128 18976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:36071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=36341
--tserver_master_addrs=127.21.236.190:38945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:37.253341 18976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:37.254455 18976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:37.264680 18989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:37.264694 18988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:37.264719 18997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:37.264948 18976 server_base.cc:1034] running on GCE node
I20260811 02:50:37.265580 18976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:37.266359 18976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:37.267604 18976 hybrid_clock.cc:648] HybridClock initialized: now 1786416637267551 us; error 63 us; skew 500 ppm
Aug 11 02:50:37 dist-test-slave-n326 krb5kdc[16663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416637, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:37.270622 18976 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:50:37.271672 18976 webserver.cc:460] Webserver started at http://127.21.236.130:36341/ using document root <none> and password file <none>
I20260811 02:50:37.272277 18976 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:37.272374 18976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:37.276309 18976 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:50:37.278564 19012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:37.280007 18976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:50:37.280176 18976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "438c5c243fee434e90ba4a65974d74a2"
format_stamp: "Formatted at 2026-08-11 02:49:54 on dist-test-slave-n326"
server_key: "69bbaa37d79619a114260b25bfffcd8c"
server_key_iv: "2f63f320da2511c9d78e11de645ce0d9"
server_key_version: "encryptionkey@0"
I20260811 02:50:37.280756 18976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:37.305075 18976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:37.308463 18976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:37.308691 18976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:37.309338 18976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:37.310588 19023 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260811 02:50:37.314720 19023 ts_tablet_manager.cc:1852] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 02:50:37.315153 19023 ts_tablet_manager.cc:1905] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:50:37.319643 19023 ts_tablet_manager.cc:1918] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:50:37.319764 19023 log.cc:1196] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/0890b3312c454171bae34e9a28c4f38f
I20260811 02:50:37.320187 19023 ts_tablet_manager.cc:1939] T 0890b3312c454171bae34e9a28c4f38f P 438c5c243fee434e90ba4a65974d74a2: Deleting consensus metadata
I20260811 02:50:37.320556 18976 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 02:50:37.320638 18976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.000s
I20260811 02:50:37.320758 18976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:37.320806 18976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:37.330796 18976 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:36071
I20260811 02:50:37.330890 19192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:36071 every 8 connection(s)
I20260811 02:50:37.331838 18976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:50:37.332099 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 18976
I20260811 02:50:37.332641 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:41589
--local_ip_for_outbound_sockets=127.21.236.131
--tserver_master_addrs=127.21.236.190:38945
--webserver_port=36075
--webserver_interface=127.21.236.131
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:50:37.336398 19197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:37.334433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:56105)
0811 02:50:37.335219 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:37.335383 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":149,"threads_started":1}
W20260811 02:50:37.336959 19193 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:38945 (0 consecutive failures): Network error: Failed to ping master at 127.21.236.190:38945: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
I20260811 02:50:37.337785 19197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:37.337499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:38117)
0811 02:50:37.337675 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:37.337727 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 02:50:37.338572 19197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:37.338326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:49601)
0811 02:50:37.338438 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:37.338502 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 02:50:37.338830 19193 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 02:50:37.450248 19195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:37.450567 19195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:37.450646 19195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:37.454675 19195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:37.454772 19195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:37.454866 19195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:50:37.460078 19195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:41589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=36075
--tserver_master_addrs=127.21.236.190:38945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:37.461319 19195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:37.462353 19195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:37.472607 19209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:37.472617 19204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:37.472666 19195 server_base.cc:1034] running on GCE node
W20260811 02:50:37.472613 19202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:37.473331 19195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:37.473992 19195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:37.475173 19195 hybrid_clock.cc:648] HybridClock initialized: now 1786416637475143 us; error 49 us; skew 500 ppm
Aug 11 02:50:37 dist-test-slave-n326 krb5kdc[16663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416637, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:37.477873 19195 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:50:37.478791 19195 webserver.cc:460] Webserver started at http://127.21.236.131:36075/ using document root <none> and password file <none>
I20260811 02:50:37.479436 19195 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:37.479537 19195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:37.483574 19195 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:50:37.486127 19217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:37.487545 19195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:50:37.487701 19195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "f9549ad6e1dc4d5aa87b832ca045a68b"
format_stamp: "Formatted at 2026-08-11 02:49:55 on dist-test-slave-n326"
server_key: "8ece1e472dacc1cac744537d8143eb52"
server_key_iv: "5efee4e16c6d5b35acd80ec81a3c9815"
server_key_version: "encryptionkey@0"
I20260811 02:50:37.488250 19195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:37.525242 19195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:37.528865 19195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:37.529114 19195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:37.529809 19195 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:37.530752 19195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:50:37.530834 19195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:37.530915 19195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:37.530973 19195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:37.542065 19195 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:41589
I20260811 02:50:37.542094 19402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:41589 every 8 connection(s)
I20260811 02:50:37.543241 19195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:50:37.546983 19406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:37.545411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:41701)
0811 02:50:37.546104 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:37.546219 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":114,"threads_started":1}
W20260811 02:50:37.547610 19403 heartbeater.cc:644] Failed to heartbeat to 127.21.236.190:38945 (0 consecutive failures): Network error: Failed to ping master at 127.21.236.190:38945: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
I20260811 02:50:37.548264 19406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:37.548054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:58933)
0811 02:50:37.548159 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:37.548209 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 02:50:37.548933 19406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:37.548735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:34963)
0811 02:50:37.548838 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:37.548882 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 02:50:37.549189 19403 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 02:50:37.552480 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 19195
I20260811 02:50:38.190210 19410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:38.189666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:37489)
0811 02:50:38.190020 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:38.190094 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":147,"threads_started":1}
I20260811 02:50:38.339922 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:38.339502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:49325)
0811 02:50:38.339775 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:38.339833 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":85,"threads_started":1}
I20260811 02:50:38.550165 19413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:38.549745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:54241)
0811 02:50:38.550010 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:38.550067 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":75,"threads_started":1}
I20260811 02:50:38.878265 19414 raft_consensus.cc:482] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:50:38.878509 19414 raft_consensus.cc:504] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:38.879710 19414 leader_election.cc:288] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589), 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071)
Aug 11 02:50:38 dist-test-slave-n326 krb5kdc[16663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416637, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.130@KRBTEST.COM
Aug 11 02:50:38 dist-test-slave-n326 krb5kdc[16663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416637, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.131@KRBTEST.COM
I20260811 02:50:38.898043 19417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:38.879974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:41589 (local address 127.21.236.129:39407)
0811 02:50:38.880380 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:38.880395 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:38.880527 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:38.882500 (+ 1973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:38.882507 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:38.883251 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:38.884150 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:38.884176 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:38.886170 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:38.886174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:38.887955 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:38.887968 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:38.888971 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:38.888975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:38.889340 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:38.890143 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:50:38.890173 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:38.892510 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:38.894943 (+ 2433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:38.894955 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:38.894957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:38.895325 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:38.895788 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:38.895791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:38.895793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:38.896031 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:38.896848 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:38.896856 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:38.897223 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 02:50:38.897452 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 02:50:38.897814 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":443,"spinlock_wait_cycles":55680,"thread_start_us":81,"threads_started":1}
I20260811 02:50:38.898137 19419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:38.880169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:32929 (local address 127.21.236.130:36071)
0811 02:50:38.880539 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:38.880557 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:38.880607 (+ 50us) server_negotiation.cc:407] Connection header received
0811 02:50:38.881556 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:38.881567 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:38.882020 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:38.882354 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:38.884367 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:38.885979 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:38.888106 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:38.888828 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:38.889011 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:38.892383 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:38.892393 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:38.892413 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:38.892449 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:38.894608 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:38.895490 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:38.895497 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:38.895503 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:38.895681 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:38.896155 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:38.896159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:38.896161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:38.896715 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:38.896854 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:38.897638 (+ 784us) server_negotiation.cc:299] Negotiation successful
0811 02:50:38.897941 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":83,"threads_started":1}
I20260811 02:50:38.898131 19418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:38.880089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:39407 (local address 127.21.236.131:41589)
0811 02:50:38.880744 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:38.880766 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:38.880798 (+ 32us) server_negotiation.cc:407] Connection header received
0811 02:50:38.881576 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:38.881583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:38.882031 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:38.882354 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:38.884367 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:38.885979 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:38.888111 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:38.888828 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:38.889011 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:38.892790 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:38.892799 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:38.892816 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:38.892859 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:38.894846 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:38.895490 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:38.895497 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:38.895503 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:38.895681 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:38.896155 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:38.896159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:38.896161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:38.896715 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:38.896854 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:38.897638 (+ 784us) server_negotiation.cc:299] Negotiation successful
0811 02:50:38.897941 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":59,"threads_started":1}
I20260811 02:50:38.898044 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:38.879966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:36071 (local address 127.21.236.129:32929)
0811 02:50:38.880313 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:38.880339 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:38.880527 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:38.882500 (+ 1973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:38.882507 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:38.883251 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:38.884150 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:38.884176 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:38.886170 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:38.886174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:38.887955 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:38.887968 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:38.888971 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:38.888975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:38.889340 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:38.890144 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:50:38.890170 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:38.892109 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:38.894807 (+ 2698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:38.894814 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:38.894834 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:38.895325 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:38.895788 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:38.895791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:38.895793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:38.896031 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:38.896848 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:38.896856 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:38.897223 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 02:50:38.897456 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 02:50:38.897874 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":833,"spinlock_wait_cycles":74880,"thread_start_us":99,"threads_started":1}
W20260811 02:50:38.900911 18783 leader_election.cc:341] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589): Not found: Tablet not found: 0890b3312c454171bae34e9a28c4f38f
W20260811 02:50:38.901113 18784 leader_election.cc:341] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071): Not found: Tablet not found: 0890b3312c454171bae34e9a28c4f38f
I20260811 02:50:38.901257 18784 leader_election.cc:302] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c7b922456034f79a691e07b8b211112; no voters: 438c5c243fee434e90ba4a65974d74a2, f9549ad6e1dc4d5aa87b832ca045a68b
I20260811 02:50:38.901568 19414 raft_consensus.cc:2726] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:50:39.191478 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:39.191127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:39467)
0811 02:50:39.191296 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:39.191371 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 02:50:39.341192 19420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:39.340735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:58253)
0811 02:50:39.341022 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:39.341083 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":75,"threads_started":1}
I20260811 02:50:39.551473 19421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:39.550973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:55491)
0811 02:50:39.551300 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:39.551375 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":142,"threads_started":1}
I20260811 02:50:40.194765 19423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:40.192405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:46743)
0811 02:50:40.194599 (+ 2194us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:40.194663 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2102,"thread_start_us":1968,"threads_started":1}
I20260811 02:50:40.342347 19424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:40.341907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:33249)
0811 02:50:40.342162 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:40.342217 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":78,"threads_started":1}
I20260811 02:50:40.514981 19425 raft_consensus.cc:482] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:50:40.515213 19425 raft_consensus.cc:504] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:40.515815 19425 leader_election.cc:288] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589), 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071)
W20260811 02:50:40.516724 18783 leader_election.cc:341] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589): Not found: Tablet not found: 0890b3312c454171bae34e9a28c4f38f
W20260811 02:50:40.516850 18784 leader_election.cc:341] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071): Not found: Tablet not found: 0890b3312c454171bae34e9a28c4f38f
I20260811 02:50:40.516954 18784 leader_election.cc:302] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c7b922456034f79a691e07b8b211112; no voters: 438c5c243fee434e90ba4a65974d74a2, f9549ad6e1dc4d5aa87b832ca045a68b
I20260811 02:50:40.517123 19425 raft_consensus.cc:2726] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:50:40.556037 19426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:40.555536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:43471)
0811 02:50:40.555827 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:40.555904 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":124,"threads_started":1}
I20260811 02:50:41.196394 19432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:41.195916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:44463)
0811 02:50:41.196227 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:41.196290 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":114,"threads_started":1}
I20260811 02:50:41.343617 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:41.343198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:45445)
0811 02:50:41.343442 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:41.343499 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":71,"threads_started":1}
I20260811 02:50:41.557315 19434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:41.556884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:58811)
0811 02:50:41.557155 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:41.557214 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":81,"threads_started":1}
I20260811 02:50:42.040095 19435 raft_consensus.cc:482] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:50:42.040237 19435 raft_consensus.cc:504] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:42.040735 19435 leader_election.cc:288] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589), 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071)
W20260811 02:50:42.041638 18784 leader_election.cc:341] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071): Not found: Tablet not found: 0890b3312c454171bae34e9a28c4f38f
W20260811 02:50:42.041728 18783 leader_election.cc:341] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589): Not found: Tablet not found: 0890b3312c454171bae34e9a28c4f38f
I20260811 02:50:42.041797 18783 leader_election.cc:302] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c7b922456034f79a691e07b8b211112; no voters: 438c5c243fee434e90ba4a65974d74a2, f9549ad6e1dc4d5aa87b832ca045a68b
I20260811 02:50:42.041952 19435 raft_consensus.cc:2726] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:50:42.199410 19436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:42.197412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:47569)
0811 02:50:42.199221 (+ 1809us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:42.199293 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1705,"thread_start_us":1455,"threads_started":1}
I20260811 02:50:42.344885 19437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:42.344443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:33365)
0811 02:50:42.344724 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:42.344787 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":83,"threads_started":1}
I20260811 02:50:42.558671 19439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:42.558217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:56053)
0811 02:50:42.558491 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:42.558575 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":81,"threads_started":1}
I20260811 02:50:43.200799 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:43.200299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:39701)
0811 02:50:43.200632 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:43.200695 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":127,"threads_started":1}
I20260811 02:50:43.346170 19442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:43.345712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:60491)
0811 02:50:43.345994 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:43.346060 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":96,"threads_started":1}
I20260811 02:50:43.559939 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:43.559535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:49763)
0811 02:50:43.559786 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:43.559843 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":81,"threads_started":1}
I20260811 02:50:43.663048 19446 raft_consensus.cc:482] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:50:43.663237 19446 raft_consensus.cc:504] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:43.663659 19446 leader_election.cc:288] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589), 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071)
W20260811 02:50:43.664464 18783 leader_election.cc:341] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589): Not found: Tablet not found: 0890b3312c454171bae34e9a28c4f38f
W20260811 02:50:43.664606 18784 leader_election.cc:341] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071): Not found: Tablet not found: 0890b3312c454171bae34e9a28c4f38f
I20260811 02:50:43.664732 18784 leader_election.cc:302] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c7b922456034f79a691e07b8b211112; no voters: 438c5c243fee434e90ba4a65974d74a2, f9549ad6e1dc4d5aa87b832ca045a68b
I20260811 02:50:43.664911 19446 raft_consensus.cc:2726] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:50:44.202540 19448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:44.201888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:52369)
0811 02:50:44.202238 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:44.202326 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":118,"threads_started":1}
I20260811 02:50:44.347419 19449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:44.346927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:54125)
0811 02:50:44.347269 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:44.347325 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":150,"threads_started":1}
I20260811 02:50:44.561200 19450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:44.560711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:53561)
0811 02:50:44.561045 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:44.561105 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":101,"threads_started":1}
I20260811 02:50:45.203881 19451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:45.203421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:55955)
0811 02:50:45.203730 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:45.203784 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":127,"threads_started":1}
I20260811 02:50:45.348742 19452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:45.348252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:56967)
0811 02:50:45.348577 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:45.348641 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":84,"threads_started":1}
I20260811 02:50:45.562671 19453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:45.562192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:59441)
0811 02:50:45.562496 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:45.562570 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":88,"threads_started":1}
I20260811 02:50:45.821478 19456 raft_consensus.cc:482] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:50:45.821619 19456 raft_consensus.cc:504] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9549ad6e1dc4d5aa87b832ca045a68b" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 41589 } } peers { permanent_uuid: "438c5c243fee434e90ba4a65974d74a2" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 36071 } } peers { permanent_uuid: "2c7b922456034f79a691e07b8b211112" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 33407 } }
I20260811 02:50:45.822129 19456 leader_election.cc:288] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589), 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071)
W20260811 02:50:45.822975 18783 leader_election.cc:341] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9549ad6e1dc4d5aa87b832ca045a68b (127.21.236.131:41589): Not found: Tablet not found: 0890b3312c454171bae34e9a28c4f38f
W20260811 02:50:45.823153 18784 leader_election.cc:341] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 438c5c243fee434e90ba4a65974d74a2 (127.21.236.130:36071): Not found: Tablet not found: 0890b3312c454171bae34e9a28c4f38f
I20260811 02:50:45.823274 18784 leader_election.cc:302] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2c7b922456034f79a691e07b8b211112; no voters: 438c5c243fee434e90ba4a65974d74a2, f9549ad6e1dc4d5aa87b832ca045a68b
I20260811 02:50:45.823462 19456 raft_consensus.cc:2726] T 0890b3312c454171bae34e9a28c4f38f P 2c7b922456034f79a691e07b8b211112 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 02:50:46.206566 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:46.204797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:39439)
0811 02:50:46.206361 (+ 1564us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:46.206443 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1472,"thread_start_us":1367,"threads_started":1}
I20260811 02:50:46.350241 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:46.349732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:50625)
0811 02:50:46.350073 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:46.350142 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":103,"threads_started":1}
I20260811 02:50:46.564210 19460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:46.563688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:43665)
0811 02:50:46.564041 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:46.564110 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":106,"threads_started":1}
I20260811 02:50:47.208101 19462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:47.207664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:53333)
0811 02:50:47.207939 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:47.207999 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":79,"threads_started":1}
I20260811 02:50:47.351522 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:47.351110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:49067)
0811 02:50:47.351372 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:47.351427 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":71,"threads_started":1}
I20260811 02:50:47.565501 19466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:47.565030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:58301)
0811 02:50:47.565345 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:47.565404 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":103,"threads_started":1}
I20260811 02:50:48.209350 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:48.208887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:48991)
0811 02:50:48.209172 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:48.209233 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":90,"threads_started":1}
I20260811 02:50:48.352733 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:48.352258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:38335)
0811 02:50:48.352583 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:48.352637 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":136,"threads_started":1}
I20260811 02:50:48.566797 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:48.566353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:52903)
0811 02:50:48.566631 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:48.566694 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":78,"threads_started":1}
I20260811 02:50:49.210651 19473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:49.210152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:41825)
0811 02:50:49.210495 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:49.210551 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":121,"threads_started":1}
I20260811 02:50:49.354033 19474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:49.353585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:57681)
0811 02:50:49.353860 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:49.353933 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":79,"threads_started":1}
I20260811 02:50:49.568125 19475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:49.567667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:55347)
0811 02:50:49.567960 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:49.568021 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":78,"threads_started":1}
I20260811 02:50:50.212070 19476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:50.211643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:35169)
0811 02:50:50.211911 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:50.211970 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":88,"threads_started":1}
I20260811 02:50:50.355470 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:50.355000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:44903)
0811 02:50:50.355312 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:50.355371 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":133,"threads_started":1}
I20260811 02:50:50.569489 19479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:50.568974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:34851)
0811 02:50:50.569310 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:50.569381 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":113,"threads_started":1}
I20260811 02:50:51.213488 19480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:51.212989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:57631)
0811 02:50:51.213317 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:51.213384 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":120,"threads_started":1}
I20260811 02:50:51.356863 19482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:51.356340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:60125)
0811 02:50:51.356674 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:51.356744 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":96,"threads_started":1}
I20260811 02:50:51.570772 19483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:51.570341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:48827)
0811 02:50:51.570615 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:51.570673 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":87,"threads_started":1}
I20260811 02:50:52.214985 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:52.214511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.129:45441)
0811 02:50:52.214800 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:52.214857 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":90,"threads_started":1}
I20260811 02:50:52.358263 19487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:52.357786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.130:55523)
0811 02:50:52.358101 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:52.358163 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":97,"threads_started":1}
I20260811 02:50:52.561225 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 18697
I20260811 02:50:52.574021 19490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:52.572565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:38945 (local address 127.21.236.131:43805)
0811 02:50:52.573817 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:52.573889 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.190:38945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1149,"thread_start_us":961,"threads_started":1}
I20260811 02:50:52.580364 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 18976
I20260811 02:50:52.590497 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 19195
2026-08-11T02:50:52Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59806 ms)
[----------] 30 tests from SecurityITest (256911 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:50:52 dist-test-slave-n326 krb5kdc[19494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:50:52 dist-test-slave-n326 krb5kdc[19494](info): set up 2 sockets
Aug 11 02:50:52 dist-test-slave-n326 krb5kdc[19494](info): commencing operation
krb5kdc: starting...
W20260811 02:50:53.952666 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.329s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:50:54 dist-test-slave-n326 krb5kdc[19494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:50:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:50:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:50:54.092218 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:46827
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:46827
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:50:54.206265 19518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:54.206593 19518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:54.206676 19518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:54.210680 19518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:50:54.210775 19518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:54.210815 19518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:54.210855 19518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:50:54.210891 19518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:50:54.216207 19518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:46827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:46827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:54.217312 19518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:54.218354 19518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:54.229590 19527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:54.229596 19524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:54.229596 19525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:54.229779 19518 server_base.cc:1034] running on GCE node
I20260811 02:50:54.230568 19518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:54.231747 19518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:54.232939 19518 hybrid_clock.cc:648] HybridClock initialized: now 1786416654232920 us; error 30 us; skew 500 ppm
Aug 11 02:50:54 dist-test-slave-n326 krb5kdc[19494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416654, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:54.235780 19518 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:50:54.236783 19518 webserver.cc:460] Webserver started at http://127.21.236.190:43663/ using document root <none> and password file <none>
I20260811 02:50:54.237378 19518 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:54.237476 19518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:54.237722 19518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:54.240018 19518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "409244889e2a4743b59d4867a915598c"
format_stamp: "Formatted at 2026-08-11 02:50:54 on dist-test-slave-n326"
server_key: "924667452c9b37d343db5a8257097494"
server_key_iv: "bd0a650e5c68ad89d8bb4190f2e577e5"
server_key_version: "encryptionkey@0"
I20260811 02:50:54.240603 19518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "409244889e2a4743b59d4867a915598c"
format_stamp: "Formatted at 2026-08-11 02:50:54 on dist-test-slave-n326"
server_key: "924667452c9b37d343db5a8257097494"
server_key_iv: "bd0a650e5c68ad89d8bb4190f2e577e5"
server_key_version: "encryptionkey@0"
I20260811 02:50:54.244704 19518 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:50:54.247262 19539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:54.248531 19518 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:50:54.248696 19518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "409244889e2a4743b59d4867a915598c"
format_stamp: "Formatted at 2026-08-11 02:50:54 on dist-test-slave-n326"
server_key: "924667452c9b37d343db5a8257097494"
server_key_iv: "bd0a650e5c68ad89d8bb4190f2e577e5"
server_key_version: "encryptionkey@0"
I20260811 02:50:54.248816 19518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:54.274610 19518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:54.277557 19518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:54.277784 19518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:54.285499 19518 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:46827
I20260811 02:50:54.285524 19631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:46827 every 8 connection(s)
I20260811 02:50:54.286434 19518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:50:54.288935 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 19518
I20260811 02:50:54.289086 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:50:54.289397 22450 external_mini_cluster.cc:1428] Setting key b86c4d6f06b11df969f170a87d235ebe
I20260811 02:50:54.289397 19632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:54.295827 19632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c: Bootstrap starting.
I20260811 02:50:54.298367 19632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c: Neither blocks nor log segments found. Creating new log.
Aug 11 02:50:54 dist-test-slave-n326 krb5kdc[19494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:50:54.299321 19632 log.cc:824] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:54.301730 19632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c: No bootstrap required, opened a new log
I20260811 02:50:54.304522 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:54.290846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55534 (local address 127.21.236.190:46827)
0811 02:50:54.291337 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:54.291352 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:54.291386 (+ 34us) server_negotiation.cc:407] Connection header received
0811 02:50:54.292173 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:54.292191 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:54.292502 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:54.292832 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:54.293637 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:54.294604 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:54.295578 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:54.296112 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:54.296287 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:54.299184 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:54.299189 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:54.299202 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:54.299230 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:54.301376 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:54.301980 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:54.301986 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:54.301991 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:54.302117 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:54.302467 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:54.302470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:54.302471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:54.302906 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:54.303157 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:54.303450 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 02:50:54.303785 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":184,"threads_started":1}
I20260811 02:50:54.305909 19632 raft_consensus.cc:348] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409244889e2a4743b59d4867a915598c" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46827 } }
I20260811 02:50:54.306190 19632 raft_consensus.cc:374] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:54.306265 19632 raft_consensus.cc:729] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 409244889e2a4743b59d4867a915598c, State: Initialized, Role: FOLLOWER
I20260811 02:50:54.306728 19632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409244889e2a4743b59d4867a915598c" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46827 } }
I20260811 02:50:54.306893 19632 raft_consensus.cc:388] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:50:54.306957 19632 raft_consensus.cc:482] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:50:54.307101 19632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:54.308100 19632 raft_consensus.cc:504] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409244889e2a4743b59d4867a915598c" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46827 } }
I20260811 02:50:54.308440 19632 leader_election.cc:302] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 409244889e2a4743b59d4867a915598c; no voters:
I20260811 02:50:54.308765 19632 leader_election.cc:288] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:50:54.308964 19638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:50:54.309314 19638 raft_consensus.cc:686] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [term 1 LEADER]: Becoming Leader. State: Replica: 409244889e2a4743b59d4867a915598c, State: Running, Role: LEADER
I20260811 02:50:54.309700 19638 consensus_queue.cc:237] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409244889e2a4743b59d4867a915598c" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46827 } }
I20260811 02:50:54.310078 19632 sys_catalog.cc:564] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:50:54.311472 19639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "409244889e2a4743b59d4867a915598c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409244889e2a4743b59d4867a915598c" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46827 } } }
I20260811 02:50:54.311623 19639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [sys.catalog]: This master's current role is: LEADER
I20260811 02:50:54.311945 19640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 409244889e2a4743b59d4867a915598c. Latest consensus state: current_term: 1 leader_uuid: "409244889e2a4743b59d4867a915598c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409244889e2a4743b59d4867a915598c" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46827 } } }
I20260811 02:50:54.312060 19640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c [sys.catalog]: This master's current role is: LEADER
W20260811 02:50:54.314877 19664 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:50:54.314951 19664 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:50:54.315040 19657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:50:54.315811 19657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:50:54.321220 19657 catalog_manager.cc:1347] Generated new cluster ID: fd545553c6c74db8a8930ba417a4784e
I20260811 02:50:54.321316 19657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:50:54.339623 19657 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:50:54.340716 19657 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:50:54.349890 19657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c: Generated new TSK 0
I20260811 02:50:54.350836 19657 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:50:54.394764 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:46827
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:50:54.513175 19678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:54.513441 19678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:54.513485 19678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:54.517601 19678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:54.517673 19678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:54.517735 19678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:50:54.523200 19678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:46827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:54.524308 19678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:54.525360 19678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:54.535187 19692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:54.535197 19695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:54.535197 19691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:54.535275 19678 server_base.cc:1034] running on GCE node
I20260811 02:50:54.535888 19678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:54.536523 19678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:54.537716 19678 hybrid_clock.cc:648] HybridClock initialized: now 1786416654537680 us; error 46 us; skew 500 ppm
Aug 11 02:50:54 dist-test-slave-n326 krb5kdc[19494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416654, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:54.540661 19678 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:50:54.541553 19678 webserver.cc:460] Webserver started at http://127.21.236.129:39941/ using document root <none> and password file <none>
I20260811 02:50:54.542191 19678 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:54.542253 19678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:54.542492 19678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:54.544530 19678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "d2cab808fa824252a81782fb3bacfd74"
format_stamp: "Formatted at 2026-08-11 02:50:54 on dist-test-slave-n326"
server_key: "cc587e8688353912b21c1fa18a61fe37"
server_key_iv: "3a658461143dbcea4a597ec9d112f85a"
server_key_version: "encryptionkey@0"
I20260811 02:50:54.545044 19678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "d2cab808fa824252a81782fb3bacfd74"
format_stamp: "Formatted at 2026-08-11 02:50:54 on dist-test-slave-n326"
server_key: "cc587e8688353912b21c1fa18a61fe37"
server_key_iv: "3a658461143dbcea4a597ec9d112f85a"
server_key_version: "encryptionkey@0"
I20260811 02:50:54.548776 19678 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:50:54.551258 19704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:54.552609 19678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:50:54.552752 19678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "d2cab808fa824252a81782fb3bacfd74"
format_stamp: "Formatted at 2026-08-11 02:50:54 on dist-test-slave-n326"
server_key: "cc587e8688353912b21c1fa18a61fe37"
server_key_iv: "3a658461143dbcea4a597ec9d112f85a"
server_key_version: "encryptionkey@0"
I20260811 02:50:54.552889 19678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:54.591887 19678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:54.594937 19678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:54.595180 19678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:54.595817 19678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:54.596698 19678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:50:54.596781 19678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:54.596859 19678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:54.596915 19678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:54.605903 19678 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:37099
I20260811 02:50:54.605934 19891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:37099 every 8 connection(s)
I20260811 02:50:54.606804 19678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:50:54.613080 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 19678
I20260811 02:50:54.613216 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:50:54.613516 22450 external_mini_cluster.cc:1428] Setting key e67254aca21f13389836358ba04bd41d
Aug 11 02:50:54 dist-test-slave-n326 krb5kdc[19494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416654, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:50:54.623036 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:54.608577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:35323 (local address 127.21.236.190:46827)
0811 02:50:54.608717 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:54.608721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:54.609499 (+ 778us) server_negotiation.cc:407] Connection header received
0811 02:50:54.611290 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:54.611296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:54.611373 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:54.611487 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:54.612838 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:54.613694 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:54.615079 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:54.615524 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:54.615657 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:54.619002 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:54.619007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:54.619011 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:54.619042 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:54.620819 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:54.621412 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:54.621416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:54.621418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:54.621513 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:54.621877 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:54.621882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:54.621884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:54.622087 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:54.622202 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:54.622715 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 02:50:54.622850 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:50:54.624001 19895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:54.608906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:46827 (local address 127.21.236.129:35323)
0811 02:50:54.609349 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:54.609391 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:54.611007 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:54.611702 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:54.611714 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:54.612115 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:54.612629 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:54.612642 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:54.613826 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:54.613829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:54.614895 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:54.614907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:54.615702 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:54.615705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:54.615854 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:54.616482 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:50:54.616499 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:54.618694 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:54.620966 (+ 2272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:54.620973 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:54.620989 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:54.621270 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:54.621620 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:54.621623 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:54.621625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:54.621775 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:54.622202 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:54.622210 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:54.622477 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 02:50:54.622674 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 02:50:54.623014 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
I20260811 02:50:54.625376 19892 heartbeater.cc:347] Connected to a master server at 127.21.236.190:46827
I20260811 02:50:54.625659 19892 heartbeater.cc:464] Registering TS with master...
I20260811 02:50:54.626219 19892 heartbeater.cc:511] Master 127.21.236.190:46827 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
I20260811 02:50:54.627938 19570 ts_manager.cc:194] Registered new tserver with Master: d2cab808fa824252a81782fb3bacfd74 (127.21.236.129:37099)
I20260811 02:50:54.629097 19570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:35323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:50:54.646996 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:46827
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:50:54.764030 19900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:54.764310 19900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:54.764356 19900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:54.768472 19900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:54.768579 19900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:54.768690 19900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:50:54.774175 19900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:46827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:54.775329 19900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:54.776361 19900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:50:54.786505 19900 server_base.cc:1034] running on GCE node
W20260811 02:50:54.786446 19917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:54.786468 19913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:54.786441 19912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:54.787158 19900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:54.787880 19900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:54.789079 19900 hybrid_clock.cc:648] HybridClock initialized: now 1786416654789064 us; error 24 us; skew 500 ppm
Aug 11 02:50:54 dist-test-slave-n326 krb5kdc[19494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416654, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:54.792238 19900 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:50:54.793200 19900 webserver.cc:460] Webserver started at http://127.21.236.130:32909/ using document root <none> and password file <none>
I20260811 02:50:54.793833 19900 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:54.793908 19900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:54.794130 19900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:54.796320 19900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "f2fe17eda8c3443fbed9b419da1107e9"
format_stamp: "Formatted at 2026-08-11 02:50:54 on dist-test-slave-n326"
server_key: "2352f4a42bcdecf059a049fa6527b4a6"
server_key_iv: "ceb5d909d53d6e8a91f26a78df414de7"
server_key_version: "encryptionkey@0"
I20260811 02:50:54.796904 19900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "f2fe17eda8c3443fbed9b419da1107e9"
format_stamp: "Formatted at 2026-08-11 02:50:54 on dist-test-slave-n326"
server_key: "2352f4a42bcdecf059a049fa6527b4a6"
server_key_iv: "ceb5d909d53d6e8a91f26a78df414de7"
server_key_version: "encryptionkey@0"
I20260811 02:50:54.800865 19900 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:50:54.803503 19929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:54.804888 19900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:50:54.805025 19900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "f2fe17eda8c3443fbed9b419da1107e9"
format_stamp: "Formatted at 2026-08-11 02:50:54 on dist-test-slave-n326"
server_key: "2352f4a42bcdecf059a049fa6527b4a6"
server_key_iv: "ceb5d909d53d6e8a91f26a78df414de7"
server_key_version: "encryptionkey@0"
I20260811 02:50:54.805141 19900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:54.823239 19900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:54.826748 19900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:54.826970 19900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:54.827652 19900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:54.828550 19900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:50:54.828624 19900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:54.828694 19900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:54.828744 19900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:54.838480 19900 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:41541
I20260811 02:50:54.838508 20143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:41541 every 8 connection(s)
I20260811 02:50:54.839545 19900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:50:54.844875 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 19900
I20260811 02:50:54.845010 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:50:54.845302 22450 external_mini_cluster.cc:1428] Setting key 0978de8e01e7c6da738a63d04f0d9e8c
Aug 11 02:50:54 dist-test-slave-n326 krb5kdc[19494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416654, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:50:54.855818 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:54.841595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:57613 (local address 127.21.236.190:46827)
0811 02:50:54.841751 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:54.841756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:54.842687 (+ 931us) server_negotiation.cc:407] Connection header received
0811 02:50:54.844092 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:54.844095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:54.844145 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:54.844224 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:54.845605 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:54.846278 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:54.847806 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:54.848201 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:54.848332 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:54.851433 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:54.851437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:54.851440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:54.851466 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:54.853294 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:54.853972 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:54.853976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:54.853978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:54.854072 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:54.854474 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:54.854478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:54.854480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:54.854697 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:54.854822 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:54.855480 (+ 658us) server_negotiation.cc:299] Negotiation successful
0811 02:50:54.855626 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 02:50:54.856871 20147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:54.842040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:46827 (local address 127.21.236.130:57613)
0811 02:50:54.842505 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:54.842574 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:54.843879 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:54.844396 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:54.844403 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:54.844813 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:54.845430 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:54.845448 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:54.846429 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:54.846433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:54.847671 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:54.847682 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:54.848377 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:54.848381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:54.848559 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:54.849259 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:50:54.849276 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:54.851201 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:54.853501 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:54.853509 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:54.853527 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:54.853839 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:54.854185 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:54.854188 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:54.854190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:54.854358 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:54.854822 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:54.854831 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:54.855197 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 02:50:54.855405 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 02:50:54.855787 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
I20260811 02:50:54.858316 20144 heartbeater.cc:347] Connected to a master server at 127.21.236.190:46827
I20260811 02:50:54.858600 20144 heartbeater.cc:464] Registering TS with master...
I20260811 02:50:54.859190 20144 heartbeater.cc:511] Master 127.21.236.190:46827 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
I20260811 02:50:54.860438 19570 ts_manager.cc:194] Registered new tserver with Master: f2fe17eda8c3443fbed9b419da1107e9 (127.21.236.130:41541)
I20260811 02:50:54.861299 19570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:57613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:50:54.879963 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:46827
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:50:54.999828 20152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:55.000141 20152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:55.000224 20152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:55.004606 20152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:55.004714 20152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:55.004810 20152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:50:55.011384 20152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:46827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:55.012676 20152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:55.013808 20152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:55.024502 20162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:55.024507 20159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:55.024529 20158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:55.024837 20152 server_base.cc:1034] running on GCE node
I20260811 02:50:55.025348 20152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:55.026098 20152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:55.027305 20152 hybrid_clock.cc:648] HybridClock initialized: now 1786416655027259 us; error 62 us; skew 500 ppm
Aug 11 02:50:55 dist-test-slave-n326 krb5kdc[19494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416655, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:55.030514 20152 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:50:55.031514 20152 webserver.cc:460] Webserver started at http://127.21.236.131:44881/ using document root <none> and password file <none>
I20260811 02:50:55.032068 20152 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:55.032128 20152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:55.032298 20152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:55.034458 20152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "1f2e440938af48f9bb86fcf96af2a81e"
format_stamp: "Formatted at 2026-08-11 02:50:55 on dist-test-slave-n326"
server_key: "cf2000a6ce691758a0ec744ef2b123e3"
server_key_iv: "7417a1f3498914eaa9f461ea30251366"
server_key_version: "encryptionkey@0"
I20260811 02:50:55.034993 20152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "1f2e440938af48f9bb86fcf96af2a81e"
format_stamp: "Formatted at 2026-08-11 02:50:55 on dist-test-slave-n326"
server_key: "cf2000a6ce691758a0ec744ef2b123e3"
server_key_iv: "7417a1f3498914eaa9f461ea30251366"
server_key_version: "encryptionkey@0"
I20260811 02:50:55.039250 20152 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:50:55.042044 20175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:55.043605 20152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:50:55.043712 20152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "1f2e440938af48f9bb86fcf96af2a81e"
format_stamp: "Formatted at 2026-08-11 02:50:55 on dist-test-slave-n326"
server_key: "cf2000a6ce691758a0ec744ef2b123e3"
server_key_iv: "7417a1f3498914eaa9f461ea30251366"
server_key_version: "encryptionkey@0"
I20260811 02:50:55.043800 20152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:55.059190 20152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:55.062690 20152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:55.062865 20152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:55.063537 20152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:55.064370 20152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:50:55.064435 20152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:55.064476 20152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:55.064502 20152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:55.073828 20152 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:44239
I20260811 02:50:55.073863 20367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:44239 every 8 connection(s)
I20260811 02:50:55.074779 20152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:50:55.078135 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 20152
I20260811 02:50:55.078274 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:50:55.078634 22450 external_mini_cluster.cc:1428] Setting key e50a2a8ce4433d728ac65e64d89b09c9
Aug 11 02:50:55 dist-test-slave-n326 krb5kdc[19494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416655, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:50:55.090179 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.076932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:51781 (local address 127.21.236.190:46827)
0811 02:50:55.077105 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:55.077109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:55.078125 (+ 1016us) server_negotiation.cc:407] Connection header received
0811 02:50:55.079866 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:55.079870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:55.079931 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:55.080011 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:55.081463 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.082015 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.083263 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.083509 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.083627 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.086287 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:55.086291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:55.086293 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:55.086323 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:55.087941 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:55.088548 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:55.088552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:55.088554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:55.088647 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:55.089010 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:55.089014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:55.089016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:55.089218 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:55.089331 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:55.089870 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 02:50:55.090012 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 02:50:55.091133 20371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.077352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:46827 (local address 127.21.236.131:51781)
0811 02:50:55.077908 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:55.077974 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:55.079608 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:55.080190 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:55.080197 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:55.080673 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:55.081288 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:55.081302 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.082154 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.082163 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:55.083129 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:55.083149 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.083683 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.083687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:55.083867 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.084489 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:50:55.084506 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:55.086097 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:55.088081 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:55.088087 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:55.088105 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:55.088398 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:55.088748 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:55.088752 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:55.088754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:55.088909 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:55.089331 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:55.089340 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:55.089615 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 02:50:55.089817 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 02:50:55.090169 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":126,"threads_started":1}
I20260811 02:50:55.092648 20369 heartbeater.cc:347] Connected to a master server at 127.21.236.190:46827
I20260811 02:50:55.092916 20369 heartbeater.cc:464] Registering TS with master...
I20260811 02:50:55.093600 20369 heartbeater.cc:511] Master 127.21.236.190:46827 requested a full tablet report, sending...
I20260811 02:50:55.094682 19570 ts_manager.cc:194] Registered new tserver with Master: 1f2e440938af48f9bb86fcf96af2a81e (127.21.236.131:44239)
I20260811 02:50:55.095255 19570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:51781
I20260811 02:50:55.105749 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:50:55.116439 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.107946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55536 (local address 127.21.236.190:46827)
0811 02:50:55.108089 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:55.108093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:55.108267 (+ 174us) server_negotiation.cc:407] Connection header received
0811 02:50:55.108368 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:55.108371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:55.108439 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:55.108594 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:55.109298 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.109998 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.110882 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.111227 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.111358 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.112959 (+ 1601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:55.112962 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:55.112965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:55.112993 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:55.114671 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:55.115166 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:55.115169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:55.115171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:55.115260 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:55.115526 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:55.115529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:55.115530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:55.115780 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:55.115900 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:55.116177 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 02:50:55.116292 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:50:55.120044 19570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:50:55.123162 19570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:50:55.138779 20384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.132282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:37099 (local address 127.0.0.1:44828)
0811 02:50:55.132980 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:55.133035 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:55.133188 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:55.133652 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:55.133659 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:55.133682 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:55.133826 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:55.133837 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.135614 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.135617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:55.137010 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:55.137019 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.138183 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.138187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:55.138310 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.138356 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 02:50:55.138482 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 02:50:55.138611 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
I20260811 02:50:55.138832 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.132513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44828 (local address 127.21.236.129:37099)
0811 02:50:55.133215 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:55.133223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:55.133251 (+ 28us) server_negotiation.cc:407] Connection header received
0811 02:50:55.133331 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:55.133335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:55.133499 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:55.133663 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:55.134113 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.135474 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.137142 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.137990 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.138167 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.138318 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:55.138549 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 02:50:55.138654 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":74,"threads_started":1}
I20260811 02:50:55.139350 20387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.132982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54156 (local address 127.21.236.130:41541)
0811 02:50:55.133504 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:55.133517 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:55.133768 (+ 251us) server_negotiation.cc:407] Connection header received
0811 02:50:55.134021 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:55.134030 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:55.134240 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:55.134383 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:55.134837 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.136065 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.137398 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.138233 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.138357 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.138486 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:55.139106 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 02:50:55.139206 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":143,"threads_started":1}
I20260811 02:50:55.139710 20385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.132359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:41541 (local address 127.0.0.1:54156)
0811 02:50:55.133662 (+ 1303us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:55.133676 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:55.133848 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:55.134600 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:55.134603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:55.134623 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:55.134710 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:55.134714 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.136239 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.136242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:55.137258 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:55.137268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.138555 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.138559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:55.138647 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.138669 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:50:55.138742 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 02:50:55.138807 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1238,"thread_start_us":130,"threads_started":1}
I20260811 02:50:55.140162 20389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.133895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:44239 (local address 127.0.0.1:58182)
0811 02:50:55.134292 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:55.134310 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:55.134393 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:55.135089 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:55.135093 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:55.135114 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:55.135204 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:55.135210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.136910 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.136913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:55.137906 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:55.137916 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.139689 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.139692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:55.139810 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.139837 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 02:50:55.139912 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 02:50:55.140005 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":123,"threads_started":1}
I20260811 02:50:55.140812 20395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.134188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58182 (local address 127.21.236.131:44239)
0811 02:50:55.134643 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:55.134653 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:55.134674 (+ 21us) server_negotiation.cc:407] Connection header received
0811 02:50:55.134746 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:55.134750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:55.134912 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:55.135055 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:55.135347 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.136747 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.138049 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.139451 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.140372 (+ 921us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.140524 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:55.140577 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 02:50:55.140668 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":159,"threads_started":1}
I20260811 02:50:55.142593 20018 tablet_service.cc:1467] Processing CreateTablet for tablet 69a15dc29489441ea277af9845d4d4c8 (DEFAULT_TABLE table=test-table [id=b6bd827c05494472a15ec29ce96d69f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:50:55.142709 19787 tablet_service.cc:1467] Processing CreateTablet for tablet 69a15dc29489441ea277af9845d4d4c8 (DEFAULT_TABLE table=test-table [id=b6bd827c05494472a15ec29ce96d69f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:50:55.143766 20018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69a15dc29489441ea277af9845d4d4c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:55.143880 19787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69a15dc29489441ea277af9845d4d4c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:55.143864 20254 tablet_service.cc:1467] Processing CreateTablet for tablet 69a15dc29489441ea277af9845d4d4c8 (DEFAULT_TABLE table=test-table [id=b6bd827c05494472a15ec29ce96d69f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:50:55.144923 20254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69a15dc29489441ea277af9845d4d4c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:55.150565 20396 tablet_bootstrap.cc:492] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9: Bootstrap starting.
I20260811 02:50:55.153128 20398 tablet_bootstrap.cc:492] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74: Bootstrap starting.
I20260811 02:50:55.153187 20400 tablet_bootstrap.cc:492] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e: Bootstrap starting.
I20260811 02:50:55.153462 20396 tablet_bootstrap.cc:654] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:55.154804 20396 log.cc:824] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:55.155187 20398 tablet_bootstrap.cc:654] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:55.155414 20400 tablet_bootstrap.cc:654] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:55.156127 20398 log.cc:824] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:55.156541 20400 log.cc:824] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:55.157317 20396 tablet_bootstrap.cc:492] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9: No bootstrap required, opened a new log
I20260811 02:50:55.157572 20396 ts_tablet_manager.cc:1397] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 02:50:55.158162 20398 tablet_bootstrap.cc:492] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74: No bootstrap required, opened a new log
I20260811 02:50:55.158351 20398 ts_tablet_manager.cc:1397] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 02:50:55.158772 20400 tablet_bootstrap.cc:492] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e: No bootstrap required, opened a new log
I20260811 02:50:55.158986 20400 ts_tablet_manager.cc:1397] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 02:50:55.161135 20396 raft_consensus.cc:348] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cab808fa824252a81782fb3bacfd74" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 37099 } } peers { permanent_uuid: "f2fe17eda8c3443fbed9b419da1107e9" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41541 } } peers { permanent_uuid: "1f2e440938af48f9bb86fcf96af2a81e" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44239 } }
I20260811 02:50:55.161425 20396 raft_consensus.cc:374] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:55.161486 20396 raft_consensus.cc:729] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2fe17eda8c3443fbed9b419da1107e9, State: Initialized, Role: FOLLOWER
I20260811 02:50:55.161988 20396 consensus_queue.cc:260] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cab808fa824252a81782fb3bacfd74" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 37099 } } peers { permanent_uuid: "f2fe17eda8c3443fbed9b419da1107e9" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41541 } } peers { permanent_uuid: "1f2e440938af48f9bb86fcf96af2a81e" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44239 } }
I20260811 02:50:55.162734 20144 heartbeater.cc:503] Master 127.21.236.190:46827 was elected leader, sending a full tablet report...
I20260811 02:50:55.162774 20398 raft_consensus.cc:348] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cab808fa824252a81782fb3bacfd74" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 37099 } } peers { permanent_uuid: "f2fe17eda8c3443fbed9b419da1107e9" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41541 } } peers { permanent_uuid: "1f2e440938af48f9bb86fcf96af2a81e" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44239 } }
I20260811 02:50:55.162774 20400 raft_consensus.cc:348] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cab808fa824252a81782fb3bacfd74" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 37099 } } peers { permanent_uuid: "f2fe17eda8c3443fbed9b419da1107e9" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41541 } } peers { permanent_uuid: "1f2e440938af48f9bb86fcf96af2a81e" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44239 } }
I20260811 02:50:55.163018 20398 raft_consensus.cc:374] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:55.163018 20400 raft_consensus.cc:374] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:55.163100 20400 raft_consensus.cc:729] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f2e440938af48f9bb86fcf96af2a81e, State: Initialized, Role: FOLLOWER
I20260811 02:50:55.163100 20398 raft_consensus.cc:729] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2cab808fa824252a81782fb3bacfd74, State: Initialized, Role: FOLLOWER
I20260811 02:50:55.163169 20396 ts_tablet_manager.cc:1428] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:50:55.163530 20400 consensus_queue.cc:260] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cab808fa824252a81782fb3bacfd74" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 37099 } } peers { permanent_uuid: "f2fe17eda8c3443fbed9b419da1107e9" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41541 } } peers { permanent_uuid: "1f2e440938af48f9bb86fcf96af2a81e" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44239 } }
I20260811 02:50:55.163540 20398 consensus_queue.cc:260] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cab808fa824252a81782fb3bacfd74" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 37099 } } peers { permanent_uuid: "f2fe17eda8c3443fbed9b419da1107e9" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41541 } } peers { permanent_uuid: "1f2e440938af48f9bb86fcf96af2a81e" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44239 } }
I20260811 02:50:55.164453 20400 ts_tablet_manager.cc:1428] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 02:50:55.164453 20398 ts_tablet_manager.cc:1428] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:50:55.164830 20369 heartbeater.cc:503] Master 127.21.236.190:46827 was elected leader, sending a full tablet report...
I20260811 02:50:55.165380 19892 heartbeater.cc:503] Master 127.21.236.190:46827 was elected leader, sending a full tablet report...
I20260811 02:50:55.279304 20407 raft_consensus.cc:482] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:50:55.279618 20407 raft_consensus.cc:504] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cab808fa824252a81782fb3bacfd74" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 37099 } } peers { permanent_uuid: "f2fe17eda8c3443fbed9b419da1107e9" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41541 } } peers { permanent_uuid: "1f2e440938af48f9bb86fcf96af2a81e" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44239 } }
I20260811 02:50:55.280823 20407 leader_election.cc:288] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2fe17eda8c3443fbed9b419da1107e9 (127.21.236.130:41541), 1f2e440938af48f9bb86fcf96af2a81e (127.21.236.131:44239)
I20260811 02:50:55.285158 20395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.281198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:54561 (local address 127.21.236.131:44239)
0811 02:50:55.281344 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:55.281351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:55.281813 (+ 462us) server_negotiation.cc:407] Connection header received
0811 02:50:55.281889 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:55.281891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:55.281966 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:55.282053 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:55.282387 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.282867 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.284115 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.284519 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.284650 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.284714 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:55.284958 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 02:50:55.285024 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:50:55.285187 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.281150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:41541 (local address 127.21.236.129:46715)
0811 02:50:55.281567 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:55.281580 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:55.281697 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:55.281985 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:55.281990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:55.282013 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:55.282203 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:55.282210 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.282989 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.282994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:55.283977 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:55.283986 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.284643 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.284646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:55.284815 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.284850 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 02:50:55.284943 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 02:50:55.285011 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":130,"thread_start_us":209,"threads_started":1}
I20260811 02:50:55.285171 20387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.281241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:46715 (local address 127.21.236.130:41541)
0811 02:50:55.281406 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:55.281411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:55.281682 (+ 271us) server_negotiation.cc:407] Connection header received
0811 02:50:55.281790 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:55.281793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:55.281854 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:55.281963 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:55.282329 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.282856 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.284113 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.284518 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.284650 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.284714 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:55.284996 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 02:50:55.285068 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:50:55.285732 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.281149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:44239 (local address 127.21.236.129:54561)
0811 02:50:55.281686 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:55.281701 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:55.281810 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:55.282073 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:55.282077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:55.282098 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:55.282218 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:55.282223 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.282984 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.282989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:55.283978 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:55.283987 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.284643 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.284646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:55.284816 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.284847 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 02:50:55.284909 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 02:50:55.285006 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":92,"thread_start_us":162,"threads_started":1}
I20260811 02:50:55.285992 20051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a15dc29489441ea277af9845d4d4c8" candidate_uuid: "d2cab808fa824252a81782fb3bacfd74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2fe17eda8c3443fbed9b419da1107e9" is_pre_election: true
I20260811 02:50:55.286295 20051 raft_consensus.cc:2454] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2cab808fa824252a81782fb3bacfd74 in term 0.
I20260811 02:50:55.286792 19705 leader_election.cc:302] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2cab808fa824252a81782fb3bacfd74, f2fe17eda8c3443fbed9b419da1107e9; no voters:
I20260811 02:50:55.287096 20407 raft_consensus.cc:2781] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:50:55.287173 20407 raft_consensus.cc:482] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:50:55.287113 20293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a15dc29489441ea277af9845d4d4c8" candidate_uuid: "d2cab808fa824252a81782fb3bacfd74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f2e440938af48f9bb86fcf96af2a81e" is_pre_election: true
I20260811 02:50:55.287213 20407 raft_consensus.cc:3037] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:55.287381 20293 raft_consensus.cc:2454] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2cab808fa824252a81782fb3bacfd74 in term 0.
I20260811 02:50:55.288506 20407 raft_consensus.cc:504] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cab808fa824252a81782fb3bacfd74" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 37099 } } peers { permanent_uuid: "f2fe17eda8c3443fbed9b419da1107e9" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41541 } } peers { permanent_uuid: "1f2e440938af48f9bb86fcf96af2a81e" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44239 } }
I20260811 02:50:55.288849 20407 leader_election.cc:288] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [CANDIDATE]: Term 1 election: Requested vote from peers f2fe17eda8c3443fbed9b419da1107e9 (127.21.236.130:41541), 1f2e440938af48f9bb86fcf96af2a81e (127.21.236.131:44239)
I20260811 02:50:55.289247 20293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a15dc29489441ea277af9845d4d4c8" candidate_uuid: "d2cab808fa824252a81782fb3bacfd74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f2e440938af48f9bb86fcf96af2a81e"
I20260811 02:50:55.289374 20293 raft_consensus.cc:3037] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:55.289404 20051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a15dc29489441ea277af9845d4d4c8" candidate_uuid: "d2cab808fa824252a81782fb3bacfd74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2fe17eda8c3443fbed9b419da1107e9"
I20260811 02:50:55.289513 20051 raft_consensus.cc:3037] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:55.290441 20293 raft_consensus.cc:2454] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2cab808fa824252a81782fb3bacfd74 in term 1.
I20260811 02:50:55.290748 20051 raft_consensus.cc:2454] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2cab808fa824252a81782fb3bacfd74 in term 1.
I20260811 02:50:55.290763 19708 leader_election.cc:302] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f2e440938af48f9bb86fcf96af2a81e, d2cab808fa824252a81782fb3bacfd74; no voters:
I20260811 02:50:55.290978 20407 raft_consensus.cc:2781] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:50:55.291260 20407 raft_consensus.cc:686] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [term 1 LEADER]: Becoming Leader. State: Replica: d2cab808fa824252a81782fb3bacfd74, State: Running, Role: LEADER
I20260811 02:50:55.291630 20407 consensus_queue.cc:237] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cab808fa824252a81782fb3bacfd74" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 37099 } } peers { permanent_uuid: "f2fe17eda8c3443fbed9b419da1107e9" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41541 } } peers { permanent_uuid: "1f2e440938af48f9bb86fcf96af2a81e" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44239 } }
I20260811 02:50:55.294692 19567 catalog_manager.cc:5495] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2cab808fa824252a81782fb3bacfd74 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "d2cab808fa824252a81782fb3bacfd74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2cab808fa824252a81782fb3bacfd74" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 37099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2fe17eda8c3443fbed9b419da1107e9" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f2e440938af48f9bb86fcf96af2a81e" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44239 } health_report { overall_health: UNKNOWN } } }
W20260811 02:50:55.325922 20370 tablet.cc:2369] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:50:55.334278 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.330192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44842 (local address 127.21.236.129:37099)
0811 02:50:55.330353 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:55.330359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:55.330376 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:50:55.330489 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:55.330492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:55.330569 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:55.330694 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:50:55.330982 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.331614 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.332603 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.332862 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.332987 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.333329 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:50:55.333810 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:50:55.333963 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:55.334033 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 02:50:55.334107 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:50:55.340561 20051 raft_consensus.cc:1261] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9 [term 1 FOLLOWER]: Refusing update from remote peer d2cab808fa824252a81782fb3bacfd74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:50:55.340610 20293 raft_consensus.cc:1261] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e [term 1 FOLLOWER]: Refusing update from remote peer d2cab808fa824252a81782fb3bacfd74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 02:50:55.340727 20146 tablet.cc:2369] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:50:55.341392 20412 consensus_queue.cc:1035] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f2e440938af48f9bb86fcf96af2a81e" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 44239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:50:55.341554 20407 consensus_queue.cc:1035] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2fe17eda8c3443fbed9b419da1107e9" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 41541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:50:55.348987 20418 mvcc.cc:205] Tried to move back new op lower bound from 7317162620267175936 to 7317162620077006848. Current Snapshot: MvccSnapshot[applied={T|T < 7317162620267175936}]
I20260811 02:50:55.350594 20420 mvcc.cc:205] Tried to move back new op lower bound from 7317162620267175936 to 7317162620077006848. Current Snapshot: MvccSnapshot[applied={T|T < 7317162620267175936}]
I20260811 02:50:55.352468 20419 mvcc.cc:205] Tried to move back new op lower bound from 7317162620267175936 to 7317162620077006848. Current Snapshot: MvccSnapshot[applied={T|T < 7317162620267175936}]
I20260811 02:50:55.358297 19565 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:50:55.358501 19565 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:50:55.361208 19565 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 409244889e2a4743b59d4867a915598c: Sending DeleteTablet for 3 replicas of tablet 69a15dc29489441ea277af9845d4d4c8
I20260811 02:50:55.362104 20018 tablet_service.cc:1514] Processing DeleteTablet for tablet 69a15dc29489441ea277af9845d4d4c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:50:55 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:54156
I20260811 02:50:55.362169 19787 tablet_service.cc:1514] Processing DeleteTablet for tablet 69a15dc29489441ea277af9845d4d4c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:50:55 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:44828
I20260811 02:50:55.362344 20254 tablet_service.cc:1514] Processing DeleteTablet for tablet 69a15dc29489441ea277af9845d4d4c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:50:55 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:58182
I20260811 02:50:55.362705 20433 tablet_replica.cc:330] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74: stopping tablet replica
I20260811 02:50:55.362922 20434 tablet_replica.cc:330] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e: stopping tablet replica
I20260811 02:50:55.363029 20433 raft_consensus.cc:2229] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:50:55.363385 20434 raft_consensus.cc:2229] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:50:55.363481 20433 raft_consensus.cc:2258] T 69a15dc29489441ea277af9845d4d4c8 P d2cab808fa824252a81782fb3bacfd74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:50:55.363595 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 19678
I20260811 02:50:55.372476 20434 raft_consensus.cc:2258] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:50:55.376744 20429 tablet_replica.cc:330] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9: stopping tablet replica
I20260811 02:50:55.376766 20434 ts_tablet_manager.cc:1905] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:50:55.377139 20429 raft_consensus.cc:2229] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:50:55.377497 20429 raft_consensus.cc:2258] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:50:55.379101 20429 ts_tablet_manager.cc:1905] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:50:55.379871 20434 ts_tablet_manager.cc:1918] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:50:55.380214 20434 log.cc:1196] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/69a15dc29489441ea277af9845d4d4c8
I20260811 02:50:55.380612 20434 ts_tablet_manager.cc:1939] T 69a15dc29489441ea277af9845d4d4c8 P 1f2e440938af48f9bb86fcf96af2a81e: Deleting consensus metadata
I20260811 02:50:55.381938 19544 catalog_manager.cc:4841] TS 1f2e440938af48f9bb86fcf96af2a81e (127.21.236.131:44239): tablet 69a15dc29489441ea277af9845d4d4c8 (table test-table [id=b6bd827c05494472a15ec29ce96d69f9]) successfully deleted
I20260811 02:50:55.382810 20429 ts_tablet_manager.cc:1918] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:50:55.382926 20429 log.cc:1196] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/69a15dc29489441ea277af9845d4d4c8
I20260811 02:50:55.383294 20429 ts_tablet_manager.cc:1939] T 69a15dc29489441ea277af9845d4d4c8 P f2fe17eda8c3443fbed9b419da1107e9: Deleting consensus metadata
I20260811 02:50:55.384714 19540 catalog_manager.cc:4841] TS f2fe17eda8c3443fbed9b419da1107e9 (127.21.236.130:41541): tablet 69a15dc29489441ea277af9845d4d4c8 (table test-table [id=b6bd827c05494472a15ec29ce96d69f9]) successfully deleted
W20260811 02:50:55.385332 19543 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:37099 (error 108)
I20260811 02:50:55.385576 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 19900
I20260811 02:50:55.389331 20389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.388923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:37099 (local address 127.0.0.1:44856)
0811 02:50:55.389121 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:55.389217 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:37099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 02:50:55.389735 19543 catalog_manager.cc:4584] TS d2cab808fa824252a81782fb3bacfd74 (127.21.236.129:37099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 69a15dc29489441ea277af9845d4d4c8: Network error: Client connection negotiation failed: client connection to 127.21.236.129:37099: connect: Connection refused (error 111)
I20260811 02:50:55.401022 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 20152
I20260811 02:50:55.413841 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 19518
2026-08-11T02:50:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2820 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T02:50:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:50:55Z Disabled control of system clock
I20260811 02:50:55.454646 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:40173
--webserver_interface=127.21.236.190
--webserver_port=0
--builtin_ntp_servers=127.21.236.148:45443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:40173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 02:50:55.574996 20440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:55.575280 20440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:55.575316 20440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:55.579341 20440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:50:55.579404 20440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:55.579419 20440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:55.579437 20440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:50:55.579452 20440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:50:55.584343 20440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:40173
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:40173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:55.585464 20440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:55.586927 20440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:55.595443 20455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:55.595467 20452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:55.595503 20448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:55.595705 20440 server_base.cc:1034] running on GCE node
I20260811 02:50:55.596395 20440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:55.597501 20440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:55.598678 20440 hybrid_clock.cc:648] HybridClock initialized: now 1786416655598648 us; error 46 us; skew 500 ppm
I20260811 02:50:55.600728 20440 webserver.cc:460] Webserver started at http://127.21.236.190:45211/ using document root <none> and password file <none>
I20260811 02:50:55.601315 20440 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:55.601392 20440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:55.601656 20440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:55.603806 20440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "0dc23b8c88ad4d9c9a5caf665499026c"
format_stamp: "Formatted at 2026-08-11 02:50:55 on dist-test-slave-n326"
server_key: "39884e15c8f5c2da2fd4025bac284de0"
server_key_iv: "c5b93339a0b882d21012e6842d1f49fd"
server_key_version: "encryptionkey@0"
I20260811 02:50:55.604385 20440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "0dc23b8c88ad4d9c9a5caf665499026c"
format_stamp: "Formatted at 2026-08-11 02:50:55 on dist-test-slave-n326"
server_key: "39884e15c8f5c2da2fd4025bac284de0"
server_key_iv: "c5b93339a0b882d21012e6842d1f49fd"
server_key_version: "encryptionkey@0"
I20260811 02:50:55.608170 20440 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:50:55.610649 20464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:55.611931 20440 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:50:55.612082 20440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "0dc23b8c88ad4d9c9a5caf665499026c"
format_stamp: "Formatted at 2026-08-11 02:50:55 on dist-test-slave-n326"
server_key: "39884e15c8f5c2da2fd4025bac284de0"
server_key_iv: "c5b93339a0b882d21012e6842d1f49fd"
server_key_version: "encryptionkey@0"
I20260811 02:50:55.612197 20440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:55.624939 20440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:55.625644 20440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:55.625847 20440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:55.633263 20440 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:40173
I20260811 02:50:55.633275 20556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:40173 every 8 connection(s)
I20260811 02:50:55.634227 20440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:50:55.637465 20558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:55.641500 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 20440
I20260811 02:50:55.641644 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:50:55.641928 22450 external_mini_cluster.cc:1428] Setting key 13a2643fe2dfe8f005fe2871860267ca
I20260811 02:50:55.643579 20558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c: Bootstrap starting.
I20260811 02:50:55.646405 20558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:55.647358 20558 log.cc:824] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:55.649474 20558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c: No bootstrap required, opened a new log
I20260811 02:50:55.653374 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.643549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34468 (local address 127.21.236.190:40173)
0811 02:50:55.644327 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:55.644350 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:55.644393 (+ 43us) server_negotiation.cc:407] Connection header received
0811 02:50:55.645300 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:55.645324 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:55.645880 (+ 556us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:55.646247 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:55.646746 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.648165 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.649048 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.649797 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.649979 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.650775 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:55.650783 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:55.650799 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:50:55.651341 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:55.651411 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:50:55.651440 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:55.651545 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:55.651684 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 02:50:55.651958 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":152,"threads_started":1}
I20260811 02:50:55.653555 20558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc23b8c88ad4d9c9a5caf665499026c" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40173 } }
I20260811 02:50:55.653811 20558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:55.653893 20558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc23b8c88ad4d9c9a5caf665499026c, State: Initialized, Role: FOLLOWER
I20260811 02:50:55.654301 20558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc23b8c88ad4d9c9a5caf665499026c" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40173 } }
I20260811 02:50:55.654428 20558 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:50:55.654520 20558 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:50:55.654618 20558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:55.655789 20558 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc23b8c88ad4d9c9a5caf665499026c" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40173 } }
I20260811 02:50:55.656126 20558 leader_election.cc:302] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dc23b8c88ad4d9c9a5caf665499026c; no voters:
I20260811 02:50:55.656459 20558 leader_election.cc:288] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:50:55.656567 20567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:50:55.656803 20567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [term 1 LEADER]: Becoming Leader. State: Replica: 0dc23b8c88ad4d9c9a5caf665499026c, State: Running, Role: LEADER
I20260811 02:50:55.657127 20567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc23b8c88ad4d9c9a5caf665499026c" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40173 } }
I20260811 02:50:55.657680 20558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:50:55.658797 20568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dc23b8c88ad4d9c9a5caf665499026c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc23b8c88ad4d9c9a5caf665499026c" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40173 } } }
I20260811 02:50:55.658942 20568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [sys.catalog]: This master's current role is: LEADER
I20260811 02:50:55.659173 20569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dc23b8c88ad4d9c9a5caf665499026c. Latest consensus state: current_term: 1 leader_uuid: "0dc23b8c88ad4d9c9a5caf665499026c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dc23b8c88ad4d9c9a5caf665499026c" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40173 } } }
I20260811 02:50:55.659271 20569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c [sys.catalog]: This master's current role is: LEADER
I20260811 02:50:55.659363 20581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:50:55.662371 20581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:50:55.668160 20581 catalog_manager.cc:1347] Generated new cluster ID: cb2ded1656bf4137952272f3dcd0df2b
I20260811 02:50:55.668231 20581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:50:55.681545 20581 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:50:55.682799 20581 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:50:55.695111 20581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c: Generated new TSK 0
I20260811 02:50:55.696000 20581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:50:55.704638 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:40173
--builtin_ntp_servers=127.21.236.148:45443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 02:50:55.822727 20598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:55.823029 20598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:55.823118 20598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:55.827159 20598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:55.827247 20598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:55.827342 20598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:50:55.832183 20598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:40173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:55.833321 20598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:55.834733 20598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:55.844988 20609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:55.845036 20604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:55.845202 20598 server_base.cc:1034] running on GCE node
W20260811 02:50:55.845208 20606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:55.845764 20598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:55.846457 20598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:55.847692 20598 hybrid_clock.cc:648] HybridClock initialized: now 1786416655847672 us; error 39 us; skew 500 ppm
I20260811 02:50:55.849975 20598 webserver.cc:460] Webserver started at http://127.21.236.129:40813/ using document root <none> and password file <none>
I20260811 02:50:55.850639 20598 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:55.850746 20598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:55.851011 20598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:55.853372 20598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "7af7c467c7564662ad63712271c80892"
format_stamp: "Formatted at 2026-08-11 02:50:55 on dist-test-slave-n326"
server_key: "c711a86dd1ded93a3a20387c84acf503"
server_key_iv: "afdd25f93eda07a6e0f233e336eb3bc0"
server_key_version: "encryptionkey@0"
I20260811 02:50:55.853996 20598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "7af7c467c7564662ad63712271c80892"
format_stamp: "Formatted at 2026-08-11 02:50:55 on dist-test-slave-n326"
server_key: "c711a86dd1ded93a3a20387c84acf503"
server_key_iv: "afdd25f93eda07a6e0f233e336eb3bc0"
server_key_version: "encryptionkey@0"
I20260811 02:50:55.858166 20598 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:50:55.860915 20619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:55.862460 20598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:50:55.862622 20598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "7af7c467c7564662ad63712271c80892"
format_stamp: "Formatted at 2026-08-11 02:50:55 on dist-test-slave-n326"
server_key: "c711a86dd1ded93a3a20387c84acf503"
server_key_iv: "afdd25f93eda07a6e0f233e336eb3bc0"
server_key_version: "encryptionkey@0"
I20260811 02:50:55.862746 20598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:55.886351 20598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:55.887198 20598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:55.887411 20598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:55.888059 20598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:55.889092 20598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:50:55.889173 20598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:55.889252 20598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:55.889310 20598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:55.899197 20598 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:34485
I20260811 02:50:55.899266 20809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:34485 every 8 connection(s)
I20260811 02:50:55.900223 20598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:50:55.902694 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 20598
I20260811 02:50:55.902832 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:50:55.903254 22450 external_mini_cluster.cc:1428] Setting key ed3b8247fbf4f310100a1256ae86df29
I20260811 02:50:55.905618 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:40173
--builtin_ntp_servers=127.21.236.148:45443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 02:50:55.911238 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.902269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:51233 (local address 127.21.236.190:40173)
0811 02:50:55.902562 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:55.902567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:55.903755 (+ 1188us) server_negotiation.cc:407] Connection header received
0811 02:50:55.905658 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:55.905661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:55.905747 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:55.905855 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:55.906918 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.907586 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.908870 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.909150 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.909274 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.910532 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:55.910540 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:55.910546 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:50:55.910567 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:55.910600 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:50:55.910615 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:55.910715 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:55.910888 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 02:50:55.910982 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260811 02:50:55.912114 20816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:55.902668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40173 (local address 127.21.236.129:51233)
0811 02:50:55.903570 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:55.903631 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:55.905409 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:55.906141 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:55.906148 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:55.906236 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:55.906765 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:55.906779 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.907738 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.907741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:55.908718 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:55.908728 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:55.909317 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:55.909322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:55.909505 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:55.910181 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:50:55.910183 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:55.910224 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:50:55.910289 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:55.910726 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:55.910734 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:55.910739 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 02:50:55.910847 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 02:50:55.911151 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":184,"threads_started":1}
I20260811 02:50:55.913480 20811 heartbeater.cc:347] Connected to a master server at 127.21.236.190:40173
I20260811 02:50:55.913769 20811 heartbeater.cc:464] Registering TS with master...
I20260811 02:50:55.914508 20811 heartbeater.cc:511] Master 127.21.236.190:40173 requested a full tablet report, sending...
I20260811 02:50:55.916375 20494 ts_manager.cc:194] Registered new tserver with Master: 7af7c467c7564662ad63712271c80892 (127.21.236.129:34485)
I20260811 02:50:55.917655 20494 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.236.129:51233
W20260811 02:50:56.043274 20817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:56.043555 20817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:56.043641 20817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:56.047600 20817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:56.047678 20817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:56.047765 20817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:50:56.052591 20817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:40173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:56.053596 20817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:56.054940 20817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:56.064654 20827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:56.064636 20832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:56.064621 20829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:56.064707 20817 server_base.cc:1034] running on GCE node
I20260811 02:50:56.065258 20817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:56.065821 20817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:56.067028 20817 hybrid_clock.cc:648] HybridClock initialized: now 1786416656067003 us; error 57 us; skew 500 ppm
I20260811 02:50:56.068756 20817 webserver.cc:460] Webserver started at http://127.21.236.130:34533/ using document root <none> and password file <none>
I20260811 02:50:56.069280 20817 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:56.069339 20817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:56.069497 20817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:56.071466 20817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "5fa0a44b95694a65940e1520700bcaae"
format_stamp: "Formatted at 2026-08-11 02:50:56 on dist-test-slave-n326"
server_key: "b1e11e99ea3f9470019626bc9622a4cd"
server_key_iv: "d9fc4e63eafc71d2fa8b765e64e29bdd"
server_key_version: "encryptionkey@0"
I20260811 02:50:56.071962 20817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "5fa0a44b95694a65940e1520700bcaae"
format_stamp: "Formatted at 2026-08-11 02:50:56 on dist-test-slave-n326"
server_key: "b1e11e99ea3f9470019626bc9622a4cd"
server_key_iv: "d9fc4e63eafc71d2fa8b765e64e29bdd"
server_key_version: "encryptionkey@0"
I20260811 02:50:56.075423 20817 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 02:50:56.077634 20843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:56.078765 20817 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:50:56.078917 20817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "5fa0a44b95694a65940e1520700bcaae"
format_stamp: "Formatted at 2026-08-11 02:50:56 on dist-test-slave-n326"
server_key: "b1e11e99ea3f9470019626bc9622a4cd"
server_key_iv: "d9fc4e63eafc71d2fa8b765e64e29bdd"
server_key_version: "encryptionkey@0"
I20260811 02:50:56.079035 20817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:56.103734 20817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:56.104399 20817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:56.104598 20817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:56.105173 20817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:56.106088 20817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:50:56.106160 20817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:56.106231 20817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:56.106285 20817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:56.115295 20817 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:33169
I20260811 02:50:56.115326 21032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:33169 every 8 connection(s)
I20260811 02:50:56.116251 20817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:50:56.125304 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 20817
I20260811 02:50:56.125327 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.118032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:45825 (local address 127.21.236.190:40173)
0811 02:50:56.118170 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:56.118174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:56.119171 (+ 997us) server_negotiation.cc:407] Connection header received
0811 02:50:56.120615 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:56.120619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:56.120680 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:56.120784 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:56.121651 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.122192 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.123382 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.123612 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.123729 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.124813 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:56.124817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:56.124819 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:50:56.124832 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:56.124861 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:50:56.124870 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:56.124946 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:56.125160 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 02:50:56.125215 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 02:50:56.125442 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:50:56.125749 22450 external_mini_cluster.cc:1428] Setting key 9bcb34b3c015be5a2bbc0c96bc088ee7
I20260811 02:50:56.126335 21036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.118368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40173 (local address 127.21.236.130:45825)
0811 02:50:56.118984 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:56.119025 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:56.120363 (+ 1338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:56.120952 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:56.120958 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:56.121038 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:56.121502 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:56.121513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.122319 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.122322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:56.123260 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:56.123269 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.123769 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.123773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:56.123938 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.124559 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:50:56.124561 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:56.124600 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:50:56.124659 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:56.124961 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:56.124973 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:56.124983 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 02:50:56.125136 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 02:50:56.125423 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":121,"threads_started":1}
I20260811 02:50:56.127507 21034 heartbeater.cc:347] Connected to a master server at 127.21.236.190:40173
I20260811 02:50:56.127764 21034 heartbeater.cc:464] Registering TS with master...
I20260811 02:50:56.127913 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:40173
--builtin_ntp_servers=127.21.236.148:45443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 02:50:56.128342 21034 heartbeater.cc:511] Master 127.21.236.190:40173 requested a full tablet report, sending...
I20260811 02:50:56.129451 20493 ts_manager.cc:194] Registered new tserver with Master: 5fa0a44b95694a65940e1520700bcaae (127.21.236.130:33169)
I20260811 02:50:56.130281 20493 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.236.130:45825
W20260811 02:50:56.251811 21038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:56.252079 21038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:56.252118 21038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:56.256034 21038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:56.256085 21038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:56.256139 21038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:50:56.261013 21038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:40173
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:56.262059 21038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:56.263409 21038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:56.272971 21053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:56.272995 21045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:56.273016 21044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:56.273305 21038 server_base.cc:1034] running on GCE node
I20260811 02:50:56.273816 21038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:56.274499 21038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:56.275693 21038 hybrid_clock.cc:648] HybridClock initialized: now 1786416656275678 us; error 25 us; skew 500 ppm
I20260811 02:50:56.277606 21038 webserver.cc:460] Webserver started at http://127.21.236.131:36877/ using document root <none> and password file <none>
I20260811 02:50:56.278223 21038 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:56.278319 21038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:56.278564 21038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:56.280717 21038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "e1560020da6b4bac8abe9807c039387c"
format_stamp: "Formatted at 2026-08-11 02:50:56 on dist-test-slave-n326"
server_key: "84245bc311cd675df8fe05b79a320db1"
server_key_iv: "92faa6066ce94af1503927aef2a3c798"
server_key_version: "encryptionkey@0"
I20260811 02:50:56.281288 21038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "e1560020da6b4bac8abe9807c039387c"
format_stamp: "Formatted at 2026-08-11 02:50:56 on dist-test-slave-n326"
server_key: "84245bc311cd675df8fe05b79a320db1"
server_key_iv: "92faa6066ce94af1503927aef2a3c798"
server_key_version: "encryptionkey@0"
I20260811 02:50:56.285188 21038 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:50:56.287745 21061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:56.289000 21038 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:50:56.289140 21038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "e1560020da6b4bac8abe9807c039387c"
format_stamp: "Formatted at 2026-08-11 02:50:56 on dist-test-slave-n326"
server_key: "84245bc311cd675df8fe05b79a320db1"
server_key_iv: "92faa6066ce94af1503927aef2a3c798"
server_key_version: "encryptionkey@0"
I20260811 02:50:56.289254 21038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:56.302266 21038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:56.302979 21038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:56.303227 21038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:56.303853 21038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:56.304822 21038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:50:56.304895 21038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:56.304965 21038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:56.305014 21038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:56.313663 21038 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:36727
I20260811 02:50:56.313721 21246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:36727 every 8 connection(s)
I20260811 02:50:56.314620 21038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:50:56.315764 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 21038
I20260811 02:50:56.315878 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:50:56.316253 22450 external_mini_cluster.cc:1428] Setting key ae0e71e93be74d77d2d42f9db018279b
I20260811 02:50:56.324733 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.316739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:41593 (local address 127.21.236.190:40173)
0811 02:50:56.316882 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:56.316887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:56.318265 (+ 1378us) server_negotiation.cc:407] Connection header received
0811 02:50:56.319870 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:56.319873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:56.319918 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:56.319987 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:56.320917 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.321484 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.322700 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.322953 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.323050 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.324235 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:56.324238 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:56.324240 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:50:56.324252 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:56.324273 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:50:56.324281 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:56.324345 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:56.324563 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 02:50:56.324617 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:50:56.325717 21253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.317215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40173 (local address 127.21.236.131:41593)
0811 02:50:56.318100 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:56.318160 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:56.319676 (+ 1516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:56.320133 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:56.320140 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:56.320230 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:56.320750 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:56.320762 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.321633 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.321636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:56.322558 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:56.322568 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.323195 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.323198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:56.323343 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.323965 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:50:56.323966 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:56.324006 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:50:56.324074 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:56.324399 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:56.324409 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:56.324415 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:50:56.324550 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 02:50:56.324832 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":141,"threads_started":1}
I20260811 02:50:56.326908 21247 heartbeater.cc:347] Connected to a master server at 127.21.236.190:40173
I20260811 02:50:56.327211 21247 heartbeater.cc:464] Registering TS with master...
I20260811 02:50:56.327802 21247 heartbeater.cc:511] Master 127.21.236.190:40173 requested a full tablet report, sending...
I20260811 02:50:56.328939 20493 ts_manager.cc:194] Registered new tserver with Master: e1560020da6b4bac8abe9807c039387c (127.21.236.131:36727)
I20260811 02:50:56.329766 20493 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.236.131:41593
I20260811 02:50:56.330555 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:50:56.336882 20565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.332639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58292 (local address 127.21.236.190:40173)
0811 02:50:56.332806 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:56.332811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:56.332991 (+ 180us) server_negotiation.cc:407] Connection header received
0811 02:50:56.333159 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:56.333162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:56.333215 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:56.333308 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:56.333625 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.334186 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.335124 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.335391 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.335526 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.336417 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:56.336429 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:56.336431 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:50:56.336444 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:56.336470 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:50:56.336479 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:56.336568 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:56.336667 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 02:50:56.336726 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:50:56.340320 20493 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:50:56.343606 20493 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:50:56.358323 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.352306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:34485 (local address 127.0.0.1:60576)
0811 02:50:56.352800 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:56.352850 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:56.353006 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:56.353971 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:56.353999 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:56.354023 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:56.354171 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:56.354184 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.355500 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.355503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:56.356646 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:56.356658 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.357795 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.357798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:56.357898 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.357933 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 02:50:56.358030 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 02:50:56.358152 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":58,"thread_start_us":70,"threads_started":1}
I20260811 02:50:56.358369 21265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.352476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60576 (local address 127.21.236.129:34485)
0811 02:50:56.353441 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:56.353454 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:56.353482 (+ 28us) server_negotiation.cc:407] Connection header received
0811 02:50:56.353579 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:56.353588 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:56.353794 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:56.354319 (+ 525us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:56.354460 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.355382 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.356801 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.357673 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.357809 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.357992 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:56.358072 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 02:50:56.358163 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":855,"thread_start_us":160,"threads_started":1}
I20260811 02:50:56.359086 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.352531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:33169 (local address 127.0.0.1:60512)
0811 02:50:56.353073 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:56.353087 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:56.353184 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:56.354478 (+ 1294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:56.354480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:56.354503 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:56.354583 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:56.354589 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.356255 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.356259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:56.357427 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:56.357438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.358629 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.358633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:56.358747 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.358772 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 02:50:56.358823 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 02:50:56.358893 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":1,"thread_start_us":109,"threads_started":1}
I20260811 02:50:56.359110 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.353052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:36727 (local address 127.0.0.1:60778)
0811 02:50:56.353461 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:56.353474 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:56.353559 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:56.354288 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:56.354294 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:56.354312 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:56.354452 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:56.354473 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.356126 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.356129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:56.357546 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:56.357557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.358752 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.358755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:56.358848 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.358867 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 02:50:56.358912 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 02:50:56.358971 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":255,"thread_start_us":123,"threads_started":1}
I20260811 02:50:56.359985 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.353047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60512 (local address 127.21.236.130:33169)
0811 02:50:56.353884 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:56.353898 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:56.353923 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:50:56.354084 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:56.354088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:56.354278 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:56.354480 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:56.354684 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.355951 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.357567 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.358440 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.359412 (+ 972us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.359637 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:56.359694 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 02:50:56.359798 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":139,"threads_started":1}
I20260811 02:50:56.360229 21270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.353413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60778 (local address 127.21.236.131:36727)
0811 02:50:56.353812 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:56.353823 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:56.353846 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:50:56.353926 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:56.353933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:56.354124 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:56.354290 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:56.354614 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.355979 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.357681 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.358621 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.359219 (+ 598us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.359808 (+ 589us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:56.359859 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 02:50:56.359955 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":100,"threads_started":1}
I20260811 02:50:56.361881 20698 tablet_service.cc:1467] Processing CreateTablet for tablet ecc6ae3d593d4a9888b6dc5526fb2dfc (DEFAULT_TABLE table=test-table [id=27421b38786541b5bf92a31c2a7d29ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:50:56.362639 20925 tablet_service.cc:1467] Processing CreateTablet for tablet ecc6ae3d593d4a9888b6dc5526fb2dfc (DEFAULT_TABLE table=test-table [id=27421b38786541b5bf92a31c2a7d29ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:50:56.363086 20698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecc6ae3d593d4a9888b6dc5526fb2dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:56.362914 21133 tablet_service.cc:1467] Processing CreateTablet for tablet ecc6ae3d593d4a9888b6dc5526fb2dfc (DEFAULT_TABLE table=test-table [id=27421b38786541b5bf92a31c2a7d29ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:50:56.363646 20925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecc6ae3d593d4a9888b6dc5526fb2dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:56.364007 21133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ecc6ae3d593d4a9888b6dc5526fb2dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:56.370498 21271 tablet_bootstrap.cc:492] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892: Bootstrap starting.
I20260811 02:50:56.370988 21272 tablet_bootstrap.cc:492] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae: Bootstrap starting.
I20260811 02:50:56.371860 21273 tablet_bootstrap.cc:492] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c: Bootstrap starting.
I20260811 02:50:56.373525 21273 tablet_bootstrap.cc:654] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:56.373566 21271 tablet_bootstrap.cc:654] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:56.373566 21272 tablet_bootstrap.cc:654] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:56.374377 21273 log.cc:824] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:56.374478 21271 log.cc:824] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:56.374468 21272 log.cc:824] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:56.376482 21273 tablet_bootstrap.cc:492] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c: No bootstrap required, opened a new log
I20260811 02:50:56.376566 21271 tablet_bootstrap.cc:492] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892: No bootstrap required, opened a new log
I20260811 02:50:56.376715 21273 ts_tablet_manager.cc:1397] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 02:50:56.376752 21271 ts_tablet_manager.cc:1397] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:50:56.377449 21272 tablet_bootstrap.cc:492] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae: No bootstrap required, opened a new log
I20260811 02:50:56.377631 21272 ts_tablet_manager.cc:1397] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260811 02:50:56.380438 21273 raft_consensus.cc:348] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af7c467c7564662ad63712271c80892" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34485 } } peers { permanent_uuid: "e1560020da6b4bac8abe9807c039387c" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36727 } } peers { permanent_uuid: "5fa0a44b95694a65940e1520700bcaae" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33169 } }
I20260811 02:50:56.380709 21273 raft_consensus.cc:374] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:56.380767 21273 raft_consensus.cc:729] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1560020da6b4bac8abe9807c039387c, State: Initialized, Role: FOLLOWER
I20260811 02:50:56.380976 21271 raft_consensus.cc:348] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af7c467c7564662ad63712271c80892" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34485 } } peers { permanent_uuid: "e1560020da6b4bac8abe9807c039387c" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36727 } } peers { permanent_uuid: "5fa0a44b95694a65940e1520700bcaae" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33169 } }
I20260811 02:50:56.381232 21271 raft_consensus.cc:374] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:56.381281 21271 raft_consensus.cc:729] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af7c467c7564662ad63712271c80892, State: Initialized, Role: FOLLOWER
I20260811 02:50:56.381326 21273 consensus_queue.cc:260] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af7c467c7564662ad63712271c80892" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34485 } } peers { permanent_uuid: "e1560020da6b4bac8abe9807c039387c" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36727 } } peers { permanent_uuid: "5fa0a44b95694a65940e1520700bcaae" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33169 } }
I20260811 02:50:56.381567 21272 raft_consensus.cc:348] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af7c467c7564662ad63712271c80892" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34485 } } peers { permanent_uuid: "e1560020da6b4bac8abe9807c039387c" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36727 } } peers { permanent_uuid: "5fa0a44b95694a65940e1520700bcaae" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33169 } }
I20260811 02:50:56.381695 21271 consensus_queue.cc:260] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af7c467c7564662ad63712271c80892" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34485 } } peers { permanent_uuid: "e1560020da6b4bac8abe9807c039387c" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36727 } } peers { permanent_uuid: "5fa0a44b95694a65940e1520700bcaae" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33169 } }
I20260811 02:50:56.381788 21272 raft_consensus.cc:374] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:56.381831 21272 raft_consensus.cc:729] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa0a44b95694a65940e1520700bcaae, State: Initialized, Role: FOLLOWER
I20260811 02:50:56.382266 21272 consensus_queue.cc:260] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af7c467c7564662ad63712271c80892" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34485 } } peers { permanent_uuid: "e1560020da6b4bac8abe9807c039387c" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36727 } } peers { permanent_uuid: "5fa0a44b95694a65940e1520700bcaae" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33169 } }
I20260811 02:50:56.382647 21273 ts_tablet_manager.cc:1428] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:50:56.382655 21271 ts_tablet_manager.cc:1428] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:50:56.383010 21247 heartbeater.cc:503] Master 127.21.236.190:40173 was elected leader, sending a full tablet report...
I20260811 02:50:56.383052 20811 heartbeater.cc:503] Master 127.21.236.190:40173 was elected leader, sending a full tablet report...
I20260811 02:50:56.383242 21272 ts_tablet_manager.cc:1428] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260811 02:50:56.383582 21034 heartbeater.cc:503] Master 127.21.236.190:40173 was elected leader, sending a full tablet report...
W20260811 02:50:56.401585 20813 tablet.cc:2369] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:50:56.403190 21278 raft_consensus.cc:482] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:50:56.403378 21278 raft_consensus.cc:504] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af7c467c7564662ad63712271c80892" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34485 } } peers { permanent_uuid: "e1560020da6b4bac8abe9807c039387c" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36727 } } peers { permanent_uuid: "5fa0a44b95694a65940e1520700bcaae" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33169 } }
I20260811 02:50:56.404551 21278 leader_election.cc:288] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1560020da6b4bac8abe9807c039387c (127.21.236.131:36727), 5fa0a44b95694a65940e1520700bcaae (127.21.236.130:33169)
I20260811 02:50:56.408556 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.404950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:53783 (local address 127.21.236.130:33169)
0811 02:50:56.405080 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:56.405087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:56.405378 (+ 291us) server_negotiation.cc:407] Connection header received
0811 02:50:56.405522 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:56.405527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:56.405577 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:56.405660 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:56.405957 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.406450 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.407601 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.407994 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.408124 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.408173 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:56.408349 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 02:50:56.408419 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:50:56.408622 20816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.404835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:36727 (local address 127.21.236.129:56325)
0811 02:50:56.405043 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:56.405058 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:56.405151 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:56.405445 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:56.405448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:56.405498 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:56.405620 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:56.405626 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.406431 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.406436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:56.407440 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:56.407451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.408165 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.408169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:56.408281 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.408299 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 02:50:56.408395 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 02:50:56.408480 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":28}
I20260811 02:50:56.408625 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.404835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:33169 (local address 127.21.236.129:53783)
0811 02:50:56.405284 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:56.405295 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:56.405408 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:56.405682 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:56.405685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:56.405701 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:56.405833 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:56.405838 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.406558 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.406563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:56.407474 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:56.407485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.408118 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.408122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:56.408215 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.408244 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 02:50:56.408332 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 02:50:56.408419 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":53,"thread_start_us":146,"threads_started":1}
I20260811 02:50:56.408840 21270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.404970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:56325 (local address 127.21.236.131:36727)
0811 02:50:56.405115 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:56.405120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:56.405138 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:50:56.405274 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:56.405276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:56.405318 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:56.405417 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:56.405744 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.406305 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.407592 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.408033 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.408171 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.408222 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:56.408413 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 02:50:56.408485 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 02:50:56.409415 21170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc6ae3d593d4a9888b6dc5526fb2dfc" candidate_uuid: "7af7c467c7564662ad63712271c80892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1560020da6b4bac8abe9807c039387c" is_pre_election: true
I20260811 02:50:56.409443 20958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc6ae3d593d4a9888b6dc5526fb2dfc" candidate_uuid: "7af7c467c7564662ad63712271c80892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa0a44b95694a65940e1520700bcaae" is_pre_election: true
I20260811 02:50:56.409710 21170 raft_consensus.cc:2454] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af7c467c7564662ad63712271c80892 in term 0.
I20260811 02:50:56.409737 20958 raft_consensus.cc:2454] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af7c467c7564662ad63712271c80892 in term 0.
I20260811 02:50:56.410262 20625 leader_election.cc:302] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5fa0a44b95694a65940e1520700bcaae, 7af7c467c7564662ad63712271c80892, e1560020da6b4bac8abe9807c039387c; no voters:
I20260811 02:50:56.410558 21278 raft_consensus.cc:2781] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:50:56.410646 21278 raft_consensus.cc:482] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:50:56.410691 21278 raft_consensus.cc:3037] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:56.411777 21278 raft_consensus.cc:504] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af7c467c7564662ad63712271c80892" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34485 } } peers { permanent_uuid: "e1560020da6b4bac8abe9807c039387c" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36727 } } peers { permanent_uuid: "5fa0a44b95694a65940e1520700bcaae" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33169 } }
I20260811 02:50:56.412111 21278 leader_election.cc:288] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [CANDIDATE]: Term 1 election: Requested vote from peers e1560020da6b4bac8abe9807c039387c (127.21.236.131:36727), 5fa0a44b95694a65940e1520700bcaae (127.21.236.130:33169)
I20260811 02:50:56.412473 20958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc6ae3d593d4a9888b6dc5526fb2dfc" candidate_uuid: "7af7c467c7564662ad63712271c80892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa0a44b95694a65940e1520700bcaae"
I20260811 02:50:56.412528 21170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ecc6ae3d593d4a9888b6dc5526fb2dfc" candidate_uuid: "7af7c467c7564662ad63712271c80892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1560020da6b4bac8abe9807c039387c"
I20260811 02:50:56.412622 20958 raft_consensus.cc:3037] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:56.412649 21170 raft_consensus.cc:3037] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:56.414023 21170 raft_consensus.cc:2454] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af7c467c7564662ad63712271c80892 in term 1.
I20260811 02:50:56.414058 20958 raft_consensus.cc:2454] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af7c467c7564662ad63712271c80892 in term 1.
I20260811 02:50:56.414350 20625 leader_election.cc:302] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fa0a44b95694a65940e1520700bcaae, 7af7c467c7564662ad63712271c80892; no voters:
I20260811 02:50:56.414536 21278 raft_consensus.cc:2781] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:50:56.414809 21278 raft_consensus.cc:686] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [term 1 LEADER]: Becoming Leader. State: Replica: 7af7c467c7564662ad63712271c80892, State: Running, Role: LEADER
I20260811 02:50:56.415261 21278 consensus_queue.cc:237] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af7c467c7564662ad63712271c80892" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34485 } } peers { permanent_uuid: "e1560020da6b4bac8abe9807c039387c" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36727 } } peers { permanent_uuid: "5fa0a44b95694a65940e1520700bcaae" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33169 } }
I20260811 02:50:56.418571 20489 catalog_manager.cc:5495] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7af7c467c7564662ad63712271c80892 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "7af7c467c7564662ad63712271c80892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af7c467c7564662ad63712271c80892" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 34485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1560020da6b4bac8abe9807c039387c" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fa0a44b95694a65940e1520700bcaae" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33169 } health_report { overall_health: UNKNOWN } } }
I20260811 02:50:56.437304 21265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.433947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60590 (local address 127.21.236.129:34485)
0811 02:50:56.434079 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:56.434083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:56.434097 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:50:56.434174 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:56.434176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:56.434231 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:56.434318 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:50:56.434564 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.435055 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.435883 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:56.436092 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:56.436170 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:56.436528 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:50:56.436929 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:50:56.437017 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:56.437150 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 02:50:56.437197 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 02:50:56.443136 20958 raft_consensus.cc:1261] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae [term 1 FOLLOWER]: Refusing update from remote peer 7af7c467c7564662ad63712271c80892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:50:56.443133 21170 raft_consensus.cc:1261] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c [term 1 FOLLOWER]: Refusing update from remote peer 7af7c467c7564662ad63712271c80892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:50:56.443871 21278 consensus_queue.cc:1035] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1560020da6b4bac8abe9807c039387c" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:50:56.444140 21284 consensus_queue.cc:1035] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 7af7c467c7564662ad63712271c80892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fa0a44b95694a65940e1520700bcaae" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:50:56.452601 21290 mvcc.cc:205] Tried to move back new op lower bound from 7317162624783933440 to 7317162624679825408. Current Snapshot: MvccSnapshot[applied={T|T < 7317162624783933440}]
I20260811 02:50:56.463721 20494 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:50:56.463886 20494 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:50:56.466270 20494 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0dc23b8c88ad4d9c9a5caf665499026c: Sending DeleteTablet for 3 replicas of tablet ecc6ae3d593d4a9888b6dc5526fb2dfc
I20260811 02:50:56.467170 20925 tablet_service.cc:1514] Processing DeleteTablet for tablet ecc6ae3d593d4a9888b6dc5526fb2dfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:50:56 UTC) from {username='slave'} at 127.0.0.1:60512
I20260811 02:50:56.467275 21133 tablet_service.cc:1514] Processing DeleteTablet for tablet ecc6ae3d593d4a9888b6dc5526fb2dfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:50:56 UTC) from {username='slave'} at 127.0.0.1:60778
I20260811 02:50:56.467356 20698 tablet_service.cc:1514] Processing DeleteTablet for tablet ecc6ae3d593d4a9888b6dc5526fb2dfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:50:56 UTC) from {username='slave'} at 127.0.0.1:60576
I20260811 02:50:56.467742 21308 tablet_replica.cc:330] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c: stopping tablet replica
I20260811 02:50:56.468086 21305 tablet_replica.cc:330] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae: stopping tablet replica
I20260811 02:50:56.468082 21308 raft_consensus.cc:2229] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:50:56.468180 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 20598
I20260811 02:50:56.468392 21308 raft_consensus.cc:2258] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:50:56.472342 21308 ts_tablet_manager.cc:1905] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:50:56.476172 21308 ts_tablet_manager.cc:1918] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:50:56.476239 21305 raft_consensus.cc:2229] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:50:56.476290 21308 log.cc:1196] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/ecc6ae3d593d4a9888b6dc5526fb2dfc
I20260811 02:50:56.476591 21305 raft_consensus.cc:2258] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:50:56.476627 21308 ts_tablet_manager.cc:1939] T ecc6ae3d593d4a9888b6dc5526fb2dfc P e1560020da6b4bac8abe9807c039387c: Deleting consensus metadata
I20260811 02:50:56.479367 21305 ts_tablet_manager.cc:1905] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:50:56.479594 20470 catalog_manager.cc:4841] TS e1560020da6b4bac8abe9807c039387c (127.21.236.131:36727): tablet ecc6ae3d593d4a9888b6dc5526fb2dfc (table test-table [id=27421b38786541b5bf92a31c2a7d29ab]) successfully deleted
I20260811 02:50:56.481678 21305 ts_tablet_manager.cc:1918] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:50:56.481801 21305 log.cc:1196] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/ecc6ae3d593d4a9888b6dc5526fb2dfc
I20260811 02:50:56.482129 21305 ts_tablet_manager.cc:1939] T ecc6ae3d593d4a9888b6dc5526fb2dfc P 5fa0a44b95694a65940e1520700bcaae: Deleting consensus metadata
I20260811 02:50:56.483318 20473 catalog_manager.cc:4841] TS 5fa0a44b95694a65940e1520700bcaae (127.21.236.130:33169): tablet ecc6ae3d593d4a9888b6dc5526fb2dfc (table test-table [id=27421b38786541b5bf92a31c2a7d29ab]) successfully deleted
W20260811 02:50:56.485890 20470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:34485 (error 108)
I20260811 02:50:56.485950 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 20817
I20260811 02:50:56.494052 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:56.493671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:34485 (local address 127.0.0.1:60606)
0811 02:50:56.493871 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:56.493945 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:34485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260811 02:50:56.494369 20470 catalog_manager.cc:4584] TS 7af7c467c7564662ad63712271c80892 (127.21.236.129:34485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ecc6ae3d593d4a9888b6dc5526fb2dfc: Network error: Client connection negotiation failed: client connection to 127.21.236.129:34485: connect: Connection refused (error 111)
I20260811 02:50:56.501426 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 21038
I20260811 02:50:56.515563 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 20440
2026-08-11T02:50:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1103 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:50:56 dist-test-slave-n326 krb5kdc[21316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:50:56 dist-test-slave-n326 krb5kdc[21316](info): set up 2 sockets
Aug 11 02:50:56 dist-test-slave-n326 krb5kdc[21316](info): commencing operation
krb5kdc: starting...
W20260811 02:50:57.832293 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:50:57 dist-test-slave-n326 krb5kdc[21316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:50:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:50:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:50:57.970793 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:39503
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:39503
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:50:58.091799 21333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:58.092080 21333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:58.092123 21333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:58.096230 21333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:50:58.096297 21333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:58.096318 21333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:58.096334 21333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:50:58.096349 21333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:50:58.101806 21333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:39503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:39503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:58.102927 21333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:58.104027 21333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:58.112584 21341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:58.112584 21340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:58.112622 21343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:58.112774 21333 server_base.cc:1034] running on GCE node
I20260811 02:50:58.113593 21333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:58.114787 21333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:58.116001 21333 hybrid_clock.cc:648] HybridClock initialized: now 1786416658115968 us; error 51 us; skew 500 ppm
Aug 11 02:50:58 dist-test-slave-n326 krb5kdc[21316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:58.128556 21333 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:50:58.129616 21333 webserver.cc:460] Webserver started at http://127.21.236.190:36275/ using document root <none> and password file <none>
I20260811 02:50:58.130241 21333 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:58.130338 21333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:58.130589 21333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:58.132786 21333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "eefc2b9be1444f389c17597636fe7b39"
format_stamp: "Formatted at 2026-08-11 02:50:58 on dist-test-slave-n326"
server_key: "42bb7185f58dbae835fe38567cb65304"
server_key_iv: "30dc101159667adeb8334461ac99d6f2"
server_key_version: "encryptionkey@0"
I20260811 02:50:58.133378 21333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "eefc2b9be1444f389c17597636fe7b39"
format_stamp: "Formatted at 2026-08-11 02:50:58 on dist-test-slave-n326"
server_key: "42bb7185f58dbae835fe38567cb65304"
server_key_iv: "30dc101159667adeb8334461ac99d6f2"
server_key_version: "encryptionkey@0"
I20260811 02:50:58.137265 21333 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 02:50:58.139771 21353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:58.140974 21333 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:50:58.141119 21333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "eefc2b9be1444f389c17597636fe7b39"
format_stamp: "Formatted at 2026-08-11 02:50:58 on dist-test-slave-n326"
server_key: "42bb7185f58dbae835fe38567cb65304"
server_key_iv: "30dc101159667adeb8334461ac99d6f2"
server_key_version: "encryptionkey@0"
I20260811 02:50:58.141234 21333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:58.166136 21333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:58.169755 21333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:58.169988 21333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:58.177539 21333 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:39503
I20260811 02:50:58.177567 21449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:39503 every 8 connection(s)
I20260811 02:50:58.178593 21333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:50:58.179337 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 21333
I20260811 02:50:58.179476 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:50:58.179786 22450 external_mini_cluster.cc:1428] Setting key 68915bafdfa790c21fd4127c569c792e
I20260811 02:50:58.181893 21450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:58.187537 21450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39: Bootstrap starting.
Aug 11 02:50:58 dist-test-slave-n326 krb5kdc[21316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:50:58.189978 21450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:58.190862 21450 log.cc:824] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:58.193256 21450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39: No bootstrap required, opened a new log
I20260811 02:50:58.195353 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:58.181216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37996 (local address 127.21.236.190:39503)
0811 02:50:58.181689 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:58.181703 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:58.181740 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:50:58.182476 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:58.182493 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:58.182814 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:58.183180 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:58.184057 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.185067 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.185952 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.186430 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.186602 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:58.189286 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:58.189291 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:58.189303 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:58.189332 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:58.191738 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:58.192274 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:58.192280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:58.192286 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:58.192418 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:58.192927 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:58.192931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:58.192933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:58.193614 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:58.193839 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:58.194164 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 02:50:58.194497 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":164,"threads_started":1}
I20260811 02:50:58.197052 21450 raft_consensus.cc:348] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefc2b9be1444f389c17597636fe7b39" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39503 } }
I20260811 02:50:58.197274 21450 raft_consensus.cc:374] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:58.197335 21450 raft_consensus.cc:729] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eefc2b9be1444f389c17597636fe7b39, State: Initialized, Role: FOLLOWER
I20260811 02:50:58.197816 21450 consensus_queue.cc:260] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefc2b9be1444f389c17597636fe7b39" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39503 } }
I20260811 02:50:58.197952 21450 raft_consensus.cc:388] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:50:58.197993 21450 raft_consensus.cc:482] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:50:58.198081 21450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:58.198966 21450 raft_consensus.cc:504] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefc2b9be1444f389c17597636fe7b39" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39503 } }
I20260811 02:50:58.199416 21450 leader_election.cc:302] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eefc2b9be1444f389c17597636fe7b39; no voters:
I20260811 02:50:58.199827 21450 leader_election.cc:288] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:50:58.200074 21458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:50:58.200379 21458 raft_consensus.cc:686] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [term 1 LEADER]: Becoming Leader. State: Replica: eefc2b9be1444f389c17597636fe7b39, State: Running, Role: LEADER
I20260811 02:50:58.200882 21458 consensus_queue.cc:237] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefc2b9be1444f389c17597636fe7b39" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39503 } }
I20260811 02:50:58.201370 21450 sys_catalog.cc:564] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:50:58.202423 21458 sys_catalog.cc:455] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eefc2b9be1444f389c17597636fe7b39. Latest consensus state: current_term: 1 leader_uuid: "eefc2b9be1444f389c17597636fe7b39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefc2b9be1444f389c17597636fe7b39" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39503 } } }
I20260811 02:50:58.202585 21458 sys_catalog.cc:458] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [sys.catalog]: This master's current role is: LEADER
I20260811 02:50:58.203405 21459 sys_catalog.cc:455] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eefc2b9be1444f389c17597636fe7b39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefc2b9be1444f389c17597636fe7b39" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39503 } } }
I20260811 02:50:58.203554 21459 sys_catalog.cc:458] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39 [sys.catalog]: This master's current role is: LEADER
W20260811 02:50:58.207379 21484 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:50:58.207472 21484 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:50:58.207548 21472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:50:58.208446 21472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:50:58.214164 21472 catalog_manager.cc:1347] Generated new cluster ID: 12df9327a1c445b798e00e2351184418
I20260811 02:50:58.214324 21472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:50:58.227628 21472 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:50:58.228663 21472 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:50:58.243918 21472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39: Generated new TSK 0
I20260811 02:50:58.244747 21472 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:50:58.282150 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39503
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:50:58.405220 21492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:58.405570 21492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:58.405651 21492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:58.409821 21492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:58.409945 21492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:58.410046 21492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:50:58.415732 21492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:58.416947 21492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:58.418044 21492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:58.428444 21498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:58.428488 21501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:58.428459 21503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:58.428686 21492 server_base.cc:1034] running on GCE node
I20260811 02:50:58.429234 21492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:58.429910 21492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:58.431239 21492 hybrid_clock.cc:648] HybridClock initialized: now 1786416658431201 us; error 53 us; skew 500 ppm
Aug 11 02:50:58 dist-test-slave-n326 krb5kdc[21316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:58.434218 21492 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:50:58.435159 21492 webserver.cc:460] Webserver started at http://127.21.236.129:36655/ using document root <none> and password file <none>
I20260811 02:50:58.435714 21492 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:58.435772 21492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:58.435935 21492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:58.437927 21492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "a3733cae8d1f43eebf8b7f7c825ecb14"
format_stamp: "Formatted at 2026-08-11 02:50:58 on dist-test-slave-n326"
server_key: "fb6e61dc6a03535eb0ca9b9da07b60c0"
server_key_iv: "67501ed40fded26d7c94b11355eb41ff"
server_key_version: "encryptionkey@0"
I20260811 02:50:58.438448 21492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "a3733cae8d1f43eebf8b7f7c825ecb14"
format_stamp: "Formatted at 2026-08-11 02:50:58 on dist-test-slave-n326"
server_key: "fb6e61dc6a03535eb0ca9b9da07b60c0"
server_key_iv: "67501ed40fded26d7c94b11355eb41ff"
server_key_version: "encryptionkey@0"
I20260811 02:50:58.442225 21492 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:50:58.444865 21515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:58.446390 21492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:50:58.446525 21492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "a3733cae8d1f43eebf8b7f7c825ecb14"
format_stamp: "Formatted at 2026-08-11 02:50:58 on dist-test-slave-n326"
server_key: "fb6e61dc6a03535eb0ca9b9da07b60c0"
server_key_iv: "67501ed40fded26d7c94b11355eb41ff"
server_key_version: "encryptionkey@0"
I20260811 02:50:58.446637 21492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:58.471891 21492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:58.475221 21492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:58.475461 21492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:58.476104 21492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:58.476980 21492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:50:58.477028 21492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:58.477099 21492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:58.477131 21492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:58.486038 21492 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:41873
I20260811 02:50:58.486070 21693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:41873 every 8 connection(s)
I20260811 02:50:58.486965 21492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:50:58.490639 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 21492
I20260811 02:50:58.490763 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:50:58.491117 22450 external_mini_cluster.cc:1428] Setting key d1444bf6402979749ae0b1b78a514aea
Aug 11 02:50:58 dist-test-slave-n326 krb5kdc[21316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
I20260811 02:50:58.504060 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:58.488859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:35473 (local address 127.21.236.190:39503)
0811 02:50:58.489007 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:58.489011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:58.489866 (+ 855us) server_negotiation.cc:407] Connection header received
0811 02:50:58.491556 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:58.491560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:58.491620 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:58.491716 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:58.493179 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.493722 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.495410 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.495827 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.496037 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:58.499271 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:58.499274 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:58.499276 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:58.499304 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:58.501237 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:58.501872 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:58.501877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:58.501879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:58.502025 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:58.502411 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:58.502417 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:58.502422 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:58.502746 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:58.502870 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:58.503737 (+ 867us) server_negotiation.cc:299] Negotiation successful
0811 02:50:58.503929 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 02:50:58.505090 21696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:58.489232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:39503 (local address 127.21.236.129:35473)
0811 02:50:58.489708 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:58.489750 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:58.491297 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:58.491918 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:58.491931 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:58.492321 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:58.493014 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:58.493034 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.493861 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.493867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:58.495283 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:58.495294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.496118 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.496122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:58.496298 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:58.497084 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:50:58.497112 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:58.499017 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:58.501382 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:58.501388 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:58.501402 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:58.501734 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:58.502137 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:58.502139 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:58.502141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:58.502283 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:58.502943 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:58.502964 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:58.503424 (+ 460us) client_negotiation.cc:769] Sending connection context
0811 02:50:58.503656 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 02:50:58.504097 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":131,"threads_started":1}
I20260811 02:50:58.506556 21694 heartbeater.cc:347] Connected to a master server at 127.21.236.190:39503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
I20260811 02:50:58.506927 21694 heartbeater.cc:464] Registering TS with master...
I20260811 02:50:58.507596 21694 heartbeater.cc:511] Master 127.21.236.190:39503 requested a full tablet report, sending...
I20260811 02:50:58.509446 21389 ts_manager.cc:194] Registered new tserver with Master: a3733cae8d1f43eebf8b7f7c825ecb14 (127.21.236.129:41873)
I20260811 02:50:58.510618 21389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:35473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:50:58.527578 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39503
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:50:58.644349 21702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:58.644663 21702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:58.644747 21702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:58.648881 21702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:58.648963 21702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:58.649053 21702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:50:58.654706 21702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:58.655838 21702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:58.656868 21702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:58.666857 21711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:58.666882 21717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:58.666862 21715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:58.667230 21702 server_base.cc:1034] running on GCE node
I20260811 02:50:58.667627 21702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:58.668308 21702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:58.669503 21702 hybrid_clock.cc:648] HybridClock initialized: now 1786416658669472 us; error 42 us; skew 500 ppm
Aug 11 02:50:58 dist-test-slave-n326 krb5kdc[21316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:58.672362 21702 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:50:58.673230 21702 webserver.cc:460] Webserver started at http://127.21.236.130:35979/ using document root <none> and password file <none>
I20260811 02:50:58.673758 21702 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:58.673818 21702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:58.673985 21702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:58.676039 21702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "f40df82f45ed4a1ebbd19e756bc39205"
format_stamp: "Formatted at 2026-08-11 02:50:58 on dist-test-slave-n326"
server_key: "c0d1e7bef3c3cd26d1b7122af1492bed"
server_key_iv: "25c5121fba6b4cb08d190a4ee4e47ff2"
server_key_version: "encryptionkey@0"
I20260811 02:50:58.676553 21702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "f40df82f45ed4a1ebbd19e756bc39205"
format_stamp: "Formatted at 2026-08-11 02:50:58 on dist-test-slave-n326"
server_key: "c0d1e7bef3c3cd26d1b7122af1492bed"
server_key_iv: "25c5121fba6b4cb08d190a4ee4e47ff2"
server_key_version: "encryptionkey@0"
I20260811 02:50:58.680217 21702 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 02:50:58.682569 21726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:58.683828 21702 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 02:50:58.683965 21702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "f40df82f45ed4a1ebbd19e756bc39205"
format_stamp: "Formatted at 2026-08-11 02:50:58 on dist-test-slave-n326"
server_key: "c0d1e7bef3c3cd26d1b7122af1492bed"
server_key_iv: "25c5121fba6b4cb08d190a4ee4e47ff2"
server_key_version: "encryptionkey@0"
I20260811 02:50:58.684069 21702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:58.696297 21702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:58.699024 21702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:58.699312 21702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:58.699890 21702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:58.700717 21702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:50:58.700765 21702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:58.700834 21702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:58.700871 21702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:58.710034 21702 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:44219
I20260811 02:50:58.710059 21910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:44219 every 8 connection(s)
I20260811 02:50:58.711004 21702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:50:58.715126 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 21702
I20260811 02:50:58.715265 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:50:58.715587 22450 external_mini_cluster.cc:1428] Setting key eafbcd94d9e9e70cfb9d3800db6301c7
Aug 11 02:50:58 dist-test-slave-n326 krb5kdc[21316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:50:58.725891 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:58.712879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:50715 (local address 127.21.236.190:39503)
0811 02:50:58.713023 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:58.713026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:58.713834 (+ 808us) server_negotiation.cc:407] Connection header received
0811 02:50:58.715202 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:58.715205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:58.715266 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:58.715378 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:58.716874 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.717429 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.718619 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.718839 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.718955 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:58.721854 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:58.721861 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:58.721867 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:58.721905 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:58.723547 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:58.724245 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:58.724248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:58.724249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:58.724325 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:58.724731 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:58.724733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:58.724734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:58.724904 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:58.724979 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:58.725663 (+ 684us) server_negotiation.cc:299] Negotiation successful
0811 02:50:58.725766 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
I20260811 02:50:58.727154 21915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:58.713243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:39503 (local address 127.21.236.130:50715)
0811 02:50:58.713684 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:58.713727 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:58.714949 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:58.715611 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:58.715623 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:58.716101 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:58.716719 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:58.716737 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.717570 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.717573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:58.718483 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:58.718493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.718998 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.719003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:58.719274 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:58.719907 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:50:58.719932 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:58.721671 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:58.723707 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:58.723716 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:58.723735 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:58.724141 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:58.724424 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:58.724427 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:58.724429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:58.724634 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:58.725011 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:58.725020 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:58.725369 (+ 349us) client_negotiation.cc:769] Sending connection context
0811 02:50:58.725619 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 02:50:58.725990 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":104,"threads_started":1}
I20260811 02:50:58.728417 21912 heartbeater.cc:347] Connected to a master server at 127.21.236.190:39503
I20260811 02:50:58.728648 21912 heartbeater.cc:464] Registering TS with master...
I20260811 02:50:58.729120 21912 heartbeater.cc:511] Master 127.21.236.190:39503 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
I20260811 02:50:58.730254 21389 ts_manager.cc:194] Registered new tserver with Master: f40df82f45ed4a1ebbd19e756bc39205 (127.21.236.130:44219)
I20260811 02:50:58.730926 21389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:50715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:50:58.747177 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39503
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:50:58.866154 21921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:58.866413 21921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:58.866456 21921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:58.870421 21921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:58.870483 21921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:58.870539 21921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:50:58.875770 21921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:58.876760 21921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:58.877702 21921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:58.887293 21926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:58.887316 21927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:58.887377 21921 server_base.cc:1034] running on GCE node
W20260811 02:50:58.887316 21929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:58.888046 21921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:58.888648 21921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:58.889815 21921 hybrid_clock.cc:648] HybridClock initialized: now 1786416658889798 us; error 34 us; skew 500 ppm
Aug 11 02:50:58 dist-test-slave-n326 krb5kdc[21316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:50:58.892916 21921 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:50:58.893744 21921 webserver.cc:460] Webserver started at http://127.21.236.131:44999/ using document root <none> and password file <none>
I20260811 02:50:58.894302 21921 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:58.894388 21921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:58.894593 21921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:58.896665 21921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "63551e46469d4a6498da42dab1d52ce2"
format_stamp: "Formatted at 2026-08-11 02:50:58 on dist-test-slave-n326"
server_key: "da625bb09637d3d6fd4bbe514b14e4ce"
server_key_iv: "1931e5e94af94d5ab4b090511e1bc21c"
server_key_version: "encryptionkey@0"
I20260811 02:50:58.897171 21921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "63551e46469d4a6498da42dab1d52ce2"
format_stamp: "Formatted at 2026-08-11 02:50:58 on dist-test-slave-n326"
server_key: "da625bb09637d3d6fd4bbe514b14e4ce"
server_key_iv: "1931e5e94af94d5ab4b090511e1bc21c"
server_key_version: "encryptionkey@0"
I20260811 02:50:58.900969 21921 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 02:50:58.903587 21942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:58.904932 21921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:50:58.905071 21921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "63551e46469d4a6498da42dab1d52ce2"
format_stamp: "Formatted at 2026-08-11 02:50:58 on dist-test-slave-n326"
server_key: "da625bb09637d3d6fd4bbe514b14e4ce"
server_key_iv: "1931e5e94af94d5ab4b090511e1bc21c"
server_key_version: "encryptionkey@0"
I20260811 02:50:58.905181 21921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:58.917299 21921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:58.920508 21921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:58.920743 21921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:58.921351 21921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:58.922199 21921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:50:58.922271 21921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:58.922343 21921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:58.922396 21921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:58.932109 21921 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:39901
I20260811 02:50:58.932144 22140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:39901 every 8 connection(s)
I20260811 02:50:58.933004 21921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:50:58.935206 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 21921
I20260811 02:50:58.935350 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:50:58.935714 22450 external_mini_cluster.cc:1428] Setting key f048719abc1df9fcd761947b613ecee4
Aug 11 02:50:58 dist-test-slave-n326 krb5kdc[21316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:50:58.947778 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:58.934928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:52197 (local address 127.21.236.190:39503)
0811 02:50:58.935136 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:58.935140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:58.936429 (+ 1289us) server_negotiation.cc:407] Connection header received
0811 02:50:58.937965 (+ 1536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:58.937969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:58.938030 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:58.938112 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:58.939414 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.940143 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.941296 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.941539 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.941654 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:58.944100 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:58.944104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:58.944106 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:58.944140 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:58.945565 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:58.946157 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:58.946160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:58.946162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:58.946246 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:58.946604 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:58.946607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:58.946609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:58.946797 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:58.946917 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:58.947475 (+ 558us) server_negotiation.cc:299] Negotiation successful
0811 02:50:58.947602 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 02:50:58.948729 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:58.935472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:39503 (local address 127.21.236.131:52197)
0811 02:50:58.936068 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:58.936128 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:58.937706 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:58.938271 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:58.938278 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:58.938654 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:58.939231 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:58.939246 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.940256 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.940265 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:58.941165 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:58.941175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.941697 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.941701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:58.941879 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:58.942423 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:50:58.942440 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:58.943935 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:58.945686 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:58.945692 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:58.945706 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:58.946049 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:58.946340 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:50:58.946343 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:50:58.946346 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:50:58.946497 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:50:58.946907 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:58.946913 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:58.947230 (+ 317us) client_negotiation.cc:769] Sending connection context
0811 02:50:58.947422 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 02:50:58.947740 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":152,"threads_started":1}
I20260811 02:50:58.950130 22141 heartbeater.cc:347] Connected to a master server at 127.21.236.190:39503
I20260811 02:50:58.950389 22141 heartbeater.cc:464] Registering TS with master...
I20260811 02:50:58.950984 22141 heartbeater.cc:511] Master 127.21.236.190:39503 requested a full tablet report, sending...
I20260811 02:50:58.952013 21387 ts_manager.cc:194] Registered new tserver with Master: 63551e46469d4a6498da42dab1d52ce2 (127.21.236.131:39901)
I20260811 02:50:58.952602 21387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:52197
I20260811 02:50:58.961980 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:50:58.973985 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:58.964089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38000 (local address 127.21.236.190:39503)
0811 02:50:58.964268 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:58.964276 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:58.964405 (+ 129us) server_negotiation.cc:407] Connection header received
0811 02:50:58.964500 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:58.964502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:58.964546 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:58.964621 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:58.965334 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.965918 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.966747 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.966996 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.967099 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:58.969044 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:58.969050 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:58.969053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:50:58.969087 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:58.971286 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:58.972073 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:58.972077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:58.972080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:58.972202 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:50:58.972657 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:50:58.972679 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:50:58.972683 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:50:58.973038 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:58.973233 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:58.973582 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 02:50:58.973722 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 02:50:58.981493 21387 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:50:58.984805 21387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:50:58.999733 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:58.993298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60802 (local address 127.21.236.129:41873)
0811 02:50:58.994504 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:58.994519 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:58.994539 (+ 20us) server_negotiation.cc:407] Connection header received
0811 02:50:58.994608 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:58.994613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:58.994761 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:58.995103 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:58.995407 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.996606 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.997986 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.998808 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.999181 (+ 373us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:58.999323 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:58.999451 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 02:50:58.999562 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1106,"thread_start_us":89,"threads_started":1}
I20260811 02:50:58.999760 22158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:58.993149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:41873 (local address 127.0.0.1:60802)
0811 02:50:58.993637 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:58.993685 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:58.993817 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:58.994901 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:58.994903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:58.994925 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:58.995307 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:58.995314 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.996756 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.996759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:58.997859 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:58.997869 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.998998 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.999001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:58.999321 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:58.999346 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 02:50:58.999416 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 02:50:58.999597 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":391,"spinlock_wait_cycles":384,"thread_start_us":88,"threads_started":1}
I20260811 02:50:58.999969 22165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:58.994298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48202 (local address 127.21.236.130:44219)
0811 02:50:58.994628 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:58.994640 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:58.994663 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:50:58.994738 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:58.994742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:58.994931 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:58.995455 (+ 524us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:58.995583 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.996287 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.997928 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.998798 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.998921 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:58.999051 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:58.999684 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 02:50:58.999816 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":131,"threads_started":1}
I20260811 02:50:59.000639 22162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:58.993846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:44219 (local address 127.0.0.1:48202)
0811 02:50:58.994117 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:58.994130 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:58.994218 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:58.995132 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:58.995135 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:58.995154 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:58.995285 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:58.995295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.996418 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.996423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:58.997776 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:58.997788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.998944 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.998949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:58.999164 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:58.999216 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 02:50:58.999340 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 02:50:58.999557 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":56,"thread_start_us":116,"threads_started":1}
I20260811 02:50:59.000927 22161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:58.993591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:39901 (local address 127.0.0.1:56730)
0811 02:50:58.993994 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:58.994008 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:58.994103 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:58.994877 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:58.994886 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:58.994911 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:58.995107 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:58.995116 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.997985 (+ 2869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.997990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:58.999317 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:58.999328 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.000494 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.000499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:59.000601 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:59.000629 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 02:50:59.000724 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 02:50:59.000796 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":131,"thread_start_us":116,"threads_started":1}
I20260811 02:50:59.001010 22163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:58.994065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56730 (local address 127.21.236.131:39901)
0811 02:50:58.994353 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:58.994366 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:58.994389 (+ 23us) server_negotiation.cc:407] Connection header received
0811 02:50:58.994478 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:58.994484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:58.994690 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:58.995346 (+ 656us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:58.995494 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:58.996614 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:58.999466 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.000339 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.000484 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:59.000609 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:59.000760 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 02:50:59.000869 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":109,"threads_started":1}
I20260811 02:50:59.003697 21592 tablet_service.cc:1467] Processing CreateTablet for tablet e7c3447fb601422e9282350dd29dc418 (DEFAULT_TABLE table=test-table [id=4246a485d05f462e8306f1433614af0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:50:59.003706 22021 tablet_service.cc:1467] Processing CreateTablet for tablet e7c3447fb601422e9282350dd29dc418 (DEFAULT_TABLE table=test-table [id=4246a485d05f462e8306f1433614af0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:50:59.003706 21803 tablet_service.cc:1467] Processing CreateTablet for tablet e7c3447fb601422e9282350dd29dc418 (DEFAULT_TABLE table=test-table [id=4246a485d05f462e8306f1433614af0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:50:59.004797 21592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7c3447fb601422e9282350dd29dc418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:59.004805 22021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7c3447fb601422e9282350dd29dc418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:59.004805 21803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7c3447fb601422e9282350dd29dc418. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:59.011955 22166 tablet_bootstrap.cc:492] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14: Bootstrap starting.
I20260811 02:50:59.013221 22168 tablet_bootstrap.cc:492] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2: Bootstrap starting.
I20260811 02:50:59.014304 22166 tablet_bootstrap.cc:654] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:59.014887 22168 tablet_bootstrap.cc:654] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:59.015195 22166 log.cc:824] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:59.015452 22169 tablet_bootstrap.cc:492] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205: Bootstrap starting.
I20260811 02:50:59.016198 22168 log.cc:824] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:59.017481 22166 tablet_bootstrap.cc:492] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14: No bootstrap required, opened a new log
I20260811 02:50:59.017563 22169 tablet_bootstrap.cc:654] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:59.017717 22166 ts_tablet_manager.cc:1397] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:50:59.018193 22168 tablet_bootstrap.cc:492] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2: No bootstrap required, opened a new log
I20260811 02:50:59.018383 22168 ts_tablet_manager.cc:1397] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 02:50:59.018797 22169 log.cc:824] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:59.020861 22169 tablet_bootstrap.cc:492] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205: No bootstrap required, opened a new log
I20260811 02:50:59.021067 22169 ts_tablet_manager.cc:1397] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:50:59.021893 22166 raft_consensus.cc:348] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41873 } } peers { permanent_uuid: "f40df82f45ed4a1ebbd19e756bc39205" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44219 } } peers { permanent_uuid: "63551e46469d4a6498da42dab1d52ce2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 39901 } }
I20260811 02:50:59.022182 22166 raft_consensus.cc:374] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:59.022243 22166 raft_consensus.cc:729] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3733cae8d1f43eebf8b7f7c825ecb14, State: Initialized, Role: FOLLOWER
I20260811 02:50:59.022120 22168 raft_consensus.cc:348] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41873 } } peers { permanent_uuid: "f40df82f45ed4a1ebbd19e756bc39205" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44219 } } peers { permanent_uuid: "63551e46469d4a6498da42dab1d52ce2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 39901 } }
I20260811 02:50:59.022369 22168 raft_consensus.cc:374] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:59.022414 22168 raft_consensus.cc:729] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63551e46469d4a6498da42dab1d52ce2, State: Initialized, Role: FOLLOWER
I20260811 02:50:59.022738 22166 consensus_queue.cc:260] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41873 } } peers { permanent_uuid: "f40df82f45ed4a1ebbd19e756bc39205" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44219 } } peers { permanent_uuid: "63551e46469d4a6498da42dab1d52ce2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 39901 } }
I20260811 02:50:59.022842 22168 consensus_queue.cc:260] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41873 } } peers { permanent_uuid: "f40df82f45ed4a1ebbd19e756bc39205" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44219 } } peers { permanent_uuid: "63551e46469d4a6498da42dab1d52ce2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 39901 } }
I20260811 02:50:59.023512 21694 heartbeater.cc:503] Master 127.21.236.190:39503 was elected leader, sending a full tablet report...
I20260811 02:50:59.023854 22168 ts_tablet_manager.cc:1428] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 02:50:59.023907 22166 ts_tablet_manager.cc:1428] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260811 02:50:59.024178 22141 heartbeater.cc:503] Master 127.21.236.190:39503 was elected leader, sending a full tablet report...
I20260811 02:50:59.026134 22169 raft_consensus.cc:348] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41873 } } peers { permanent_uuid: "f40df82f45ed4a1ebbd19e756bc39205" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44219 } } peers { permanent_uuid: "63551e46469d4a6498da42dab1d52ce2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 39901 } }
I20260811 02:50:59.026418 22169 raft_consensus.cc:374] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:59.026504 22169 raft_consensus.cc:729] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f40df82f45ed4a1ebbd19e756bc39205, State: Initialized, Role: FOLLOWER
I20260811 02:50:59.027515 22169 consensus_queue.cc:260] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41873 } } peers { permanent_uuid: "f40df82f45ed4a1ebbd19e756bc39205" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44219 } } peers { permanent_uuid: "63551e46469d4a6498da42dab1d52ce2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 39901 } }
I20260811 02:50:59.028239 21912 heartbeater.cc:503] Master 127.21.236.190:39503 was elected leader, sending a full tablet report...
I20260811 02:50:59.029552 22169 ts_tablet_manager.cc:1428] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 02:50:59.064731 22173 raft_consensus.cc:482] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:50:59.064927 22173 raft_consensus.cc:504] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41873 } } peers { permanent_uuid: "f40df82f45ed4a1ebbd19e756bc39205" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44219 } } peers { permanent_uuid: "63551e46469d4a6498da42dab1d52ce2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 39901 } }
I20260811 02:50:59.066075 22173 leader_election.cc:288] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f40df82f45ed4a1ebbd19e756bc39205 (127.21.236.130:44219), 63551e46469d4a6498da42dab1d52ce2 (127.21.236.131:39901)
I20260811 02:50:59.070448 22163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:59.066553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:53007 (local address 127.21.236.131:39901)
0811 02:50:59.066699 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:59.066703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:59.066863 (+ 160us) server_negotiation.cc:407] Connection header received
0811 02:50:59.067139 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:59.067142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:59.067241 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:59.067331 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:59.067715 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.068262 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.069431 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.069822 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.069920 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:59.070005 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:59.070211 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 02:50:59.070290 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 02:50:59.070477 22165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:59.066411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:54003 (local address 127.21.236.130:44219)
0811 02:50:59.066577 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:59.066582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:59.066830 (+ 248us) server_negotiation.cc:407] Connection header received
0811 02:50:59.067006 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:59.067010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:59.067115 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:59.067189 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:50:59.067555 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.068167 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.069393 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.069845 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.070004 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:59.070062 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:59.070255 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 02:50:59.070336 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 02:50:59.070509 22178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:59.066287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:44219 (local address 127.21.236.129:54003)
0811 02:50:59.066704 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:59.066719 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:59.066867 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:59.067243 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:59.067247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:59.067266 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:59.067416 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:59.067426 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.068299 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.068306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:59.069259 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:59.069269 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.070014 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.070021 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:59.070107 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:59.070125 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 02:50:59.070221 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 02:50:59.070294 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":36,"thread_start_us":155,"threads_started":1}
I20260811 02:50:59.070726 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:59.066393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:39901 (local address 127.21.236.129:53007)
0811 02:50:59.066733 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:59.066746 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:59.066904 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:59.067367 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:59.067369 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:59.067385 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:50:59.067568 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:59.067576 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.068388 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.068403 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:59.069266 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:59.069275 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.069959 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.069963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:59.070061 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:59.070094 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 02:50:59.070170 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 02:50:59.070269 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":2,"thread_start_us":136,"threads_started":1}
I20260811 02:50:59.071225 21836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7c3447fb601422e9282350dd29dc418" candidate_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f40df82f45ed4a1ebbd19e756bc39205" is_pre_election: true
I20260811 02:50:59.071416 22052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7c3447fb601422e9282350dd29dc418" candidate_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63551e46469d4a6498da42dab1d52ce2" is_pre_election: true
I20260811 02:50:59.071573 21836 raft_consensus.cc:2454] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3733cae8d1f43eebf8b7f7c825ecb14 in term 0.
I20260811 02:50:59.071729 22052 raft_consensus.cc:2454] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3733cae8d1f43eebf8b7f7c825ecb14 in term 0.
I20260811 02:50:59.072151 21518 leader_election.cc:302] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3733cae8d1f43eebf8b7f7c825ecb14, f40df82f45ed4a1ebbd19e756bc39205; no voters:
I20260811 02:50:59.072504 22173 raft_consensus.cc:2781] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:50:59.072602 22173 raft_consensus.cc:482] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:50:59.072650 22173 raft_consensus.cc:3037] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:59.073768 22173 raft_consensus.cc:504] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41873 } } peers { permanent_uuid: "f40df82f45ed4a1ebbd19e756bc39205" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44219 } } peers { permanent_uuid: "63551e46469d4a6498da42dab1d52ce2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 39901 } }
I20260811 02:50:59.074141 22173 leader_election.cc:288] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [CANDIDATE]: Term 1 election: Requested vote from peers f40df82f45ed4a1ebbd19e756bc39205 (127.21.236.130:44219), 63551e46469d4a6498da42dab1d52ce2 (127.21.236.131:39901)
I20260811 02:50:59.074476 21836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7c3447fb601422e9282350dd29dc418" candidate_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f40df82f45ed4a1ebbd19e756bc39205"
I20260811 02:50:59.074590 22052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7c3447fb601422e9282350dd29dc418" candidate_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63551e46469d4a6498da42dab1d52ce2"
I20260811 02:50:59.074647 21836 raft_consensus.cc:3037] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:59.074776 22052 raft_consensus.cc:3037] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:59.075948 22052 raft_consensus.cc:2454] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3733cae8d1f43eebf8b7f7c825ecb14 in term 1.
I20260811 02:50:59.075968 21836 raft_consensus.cc:2454] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3733cae8d1f43eebf8b7f7c825ecb14 in term 1.
I20260811 02:50:59.076304 21516 leader_election.cc:302] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63551e46469d4a6498da42dab1d52ce2, a3733cae8d1f43eebf8b7f7c825ecb14; no voters:
I20260811 02:50:59.076524 22173 raft_consensus.cc:2781] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:50:59.076764 22173 raft_consensus.cc:686] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [term 1 LEADER]: Becoming Leader. State: Replica: a3733cae8d1f43eebf8b7f7c825ecb14, State: Running, Role: LEADER
I20260811 02:50:59.077162 22173 consensus_queue.cc:237] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41873 } } peers { permanent_uuid: "f40df82f45ed4a1ebbd19e756bc39205" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44219 } } peers { permanent_uuid: "63551e46469d4a6498da42dab1d52ce2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 39901 } }
I20260811 02:50:59.080458 21387 catalog_manager.cc:5495] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3733cae8d1f43eebf8b7f7c825ecb14 (127.21.236.129). New cstate: current_term: 1 leader_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3733cae8d1f43eebf8b7f7c825ecb14" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f40df82f45ed4a1ebbd19e756bc39205" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63551e46469d4a6498da42dab1d52ce2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 39901 } health_report { overall_health: UNKNOWN } } }
I20260811 02:50:59.100762 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:59.096470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60814 (local address 127.21.236.129:41873)
0811 02:50:59.096715 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:59.096721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:59.096804 (+ 83us) server_negotiation.cc:407] Connection header received
0811 02:50:59.097035 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:59.097038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:59.097100 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:59.097227 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:50:59.097544 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.098132 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.099142 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.099387 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.099514 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:59.099841 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:50:59.100273 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:50:59.100448 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:59.100566 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 02:50:59.100612 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260811 02:50:59.107024 22052 raft_consensus.cc:1261] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2 [term 1 FOLLOWER]: Refusing update from remote peer a3733cae8d1f43eebf8b7f7c825ecb14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:50:59.107697 21836 raft_consensus.cc:1261] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205 [term 1 FOLLOWER]: Refusing update from remote peer a3733cae8d1f43eebf8b7f7c825ecb14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:50:59.107880 22173 consensus_queue.cc:1035] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63551e46469d4a6498da42dab1d52ce2" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 39901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:50:59.108378 22185 consensus_queue.cc:1035] T e7c3447fb601422e9282350dd29dc418 P a3733cae8d1f43eebf8b7f7c825ecb14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f40df82f45ed4a1ebbd19e756bc39205" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 44219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:50:59.117630 22191 mvcc.cc:205] Tried to move back new op lower bound from 7317162635694616576 to 7317162635582906368. Current Snapshot: MvccSnapshot[applied={T|T < 7317162635694616576}]
I20260811 02:50:59.117848 22192 mvcc.cc:205] Tried to move back new op lower bound from 7317162635694616576 to 7317162635582906368. Current Snapshot: MvccSnapshot[applied={T|T < 7317162635694616576}]
I20260811 02:50:59.126416 21387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:50:59.126571 21387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:50:59.129223 21387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eefc2b9be1444f389c17597636fe7b39: Sending DeleteTablet for 3 replicas of tablet e7c3447fb601422e9282350dd29dc418
I20260811 02:50:59.130081 21803 tablet_service.cc:1514] Processing DeleteTablet for tablet e7c3447fb601422e9282350dd29dc418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:50:59 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:48202
I20260811 02:50:59.130082 22021 tablet_service.cc:1514] Processing DeleteTablet for tablet e7c3447fb601422e9282350dd29dc418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:50:59 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:56730
I20260811 02:50:59.130268 21592 tablet_service.cc:1514] Processing DeleteTablet for tablet e7c3447fb601422e9282350dd29dc418 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:50:59 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:60802
I20260811 02:50:59.130894 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 21492
I20260811 02:50:59.137830 22203 tablet_replica.cc:330] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2: stopping tablet replica
I20260811 02:50:59.138283 22203 raft_consensus.cc:2229] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:50:59.138722 22203 raft_consensus.cc:2258] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:50:59.139178 22204 tablet_replica.cc:330] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205: stopping tablet replica
I20260811 02:50:59.139569 22204 raft_consensus.cc:2229] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:50:59.139942 22204 raft_consensus.cc:2258] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:50:59.142827 22204 ts_tablet_manager.cc:1905] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:50:59.142951 22203 ts_tablet_manager.cc:1905] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:50:59.145673 22203 ts_tablet_manager.cc:1918] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:50:59.145804 22203 log.cc:1196] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/e7c3447fb601422e9282350dd29dc418
I20260811 02:50:59.146188 22203 ts_tablet_manager.cc:1939] T e7c3447fb601422e9282350dd29dc418 P 63551e46469d4a6498da42dab1d52ce2: Deleting consensus metadata
I20260811 02:50:59.146427 22204 ts_tablet_manager.cc:1918] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:50:59.146559 22204 log.cc:1196] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/e7c3447fb601422e9282350dd29dc418
I20260811 02:50:59.146911 22204 ts_tablet_manager.cc:1939] T e7c3447fb601422e9282350dd29dc418 P f40df82f45ed4a1ebbd19e756bc39205: Deleting consensus metadata
I20260811 02:50:59.147758 21356 catalog_manager.cc:4841] TS 63551e46469d4a6498da42dab1d52ce2 (127.21.236.131:39901): tablet e7c3447fb601422e9282350dd29dc418 (table test-table [id=4246a485d05f462e8306f1433614af0a]) successfully deleted
I20260811 02:50:59.148150 21359 catalog_manager.cc:4841] TS f40df82f45ed4a1ebbd19e756bc39205 (127.21.236.130:44219): tablet e7c3447fb601422e9282350dd29dc418 (table test-table [id=4246a485d05f462e8306f1433614af0a]) successfully deleted
W20260811 02:50:59.149083 21359 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:41873 (error 108)
I20260811 02:50:59.149374 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 21702
I20260811 02:50:59.157912 22162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:59.157540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:41873 (local address 127.0.0.1:60824)
0811 02:50:59.157697 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:59.157789 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 02:50:59.158115 21359 catalog_manager.cc:4584] TS a3733cae8d1f43eebf8b7f7c825ecb14 (127.21.236.129:41873): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e7c3447fb601422e9282350dd29dc418: Network error: Client connection negotiation failed: client connection to 127.21.236.129:41873: connect: Connection refused (error 111)
I20260811 02:50:59.175451 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 21921
I20260811 02:50:59.193089 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 21333
2026-08-11T02:50:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2677 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T02:50:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:50:59Z Disabled control of system clock
I20260811 02:50:59.235721 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:45439
--webserver_interface=127.21.236.190
--webserver_port=0
--builtin_ntp_servers=127.21.236.148:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:45439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 02:50:59.353200 22215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:59.353487 22215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:59.353523 22215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:59.357805 22215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:50:59.357872 22215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:59.357888 22215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:59.357905 22215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:50:59.357920 22215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:50:59.362777 22215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:45439
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:45439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:59.363953 22215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:59.365396 22215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:59.374219 22226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:59.374260 22222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:59.374439 22215 server_base.cc:1034] running on GCE node
W20260811 02:50:59.374277 22223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:59.375221 22215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:59.376286 22215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:59.377454 22215 hybrid_clock.cc:648] HybridClock initialized: now 1786416659377435 us; error 30 us; skew 500 ppm
I20260811 02:50:59.379374 22215 webserver.cc:460] Webserver started at http://127.21.236.190:33937/ using document root <none> and password file <none>
I20260811 02:50:59.379932 22215 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:59.379995 22215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:59.380177 22215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:59.382267 22215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "adec2c93cb1643acb4b917e36fd27ec5"
format_stamp: "Formatted at 2026-08-11 02:50:59 on dist-test-slave-n326"
server_key: "0d47d7e1340811918a7d55d7debcf557"
server_key_iv: "7e02584685c705f65e1220a5427b8f13"
server_key_version: "encryptionkey@0"
I20260811 02:50:59.382823 22215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "adec2c93cb1643acb4b917e36fd27ec5"
format_stamp: "Formatted at 2026-08-11 02:50:59 on dist-test-slave-n326"
server_key: "0d47d7e1340811918a7d55d7debcf557"
server_key_iv: "7e02584685c705f65e1220a5427b8f13"
server_key_version: "encryptionkey@0"
I20260811 02:50:59.386729 22215 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:50:59.389451 22236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:59.391222 22215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:50:59.391345 22215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "adec2c93cb1643acb4b917e36fd27ec5"
format_stamp: "Formatted at 2026-08-11 02:50:59 on dist-test-slave-n326"
server_key: "0d47d7e1340811918a7d55d7debcf557"
server_key_iv: "7e02584685c705f65e1220a5427b8f13"
server_key_version: "encryptionkey@0"
I20260811 02:50:59.391456 22215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:59.410455 22215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:59.411273 22215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:59.411463 22215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:59.419149 22215 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:45439
I20260811 02:50:59.419217 22318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:45439 every 8 connection(s)
I20260811 02:50:59.420323 22215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:50:59.422551 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 22215
I20260811 02:50:59.422709 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:50:59.423036 22450 external_mini_cluster.cc:1428] Setting key 276dfdcb1e223bbba0577ffdf496df7d
I20260811 02:50:59.424409 22319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:50:59.431027 22319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5: Bootstrap starting.
I20260811 02:50:59.432839 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:59.424745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39198 (local address 127.21.236.190:45439)
0811 02:50:59.425431 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:59.425448 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:59.425485 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:50:59.426157 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:59.426173 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:59.426580 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:59.426874 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:59.427297 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.428270 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.429478 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.429949 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.430070 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:59.430964 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:59.430969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:59.430981 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:50:59.431498 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:59.431561 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:50:59.431583 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:59.431659 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:59.431831 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 02:50:59.432126 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":164,"threads_started":1}
I20260811 02:50:59.433537 22319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5: Neither blocks nor log segments found. Creating new log.
I20260811 02:50:59.434350 22319 log.cc:824] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5: Log is configured to *not* fsync() on all Append() calls
I20260811 02:50:59.436368 22319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5: No bootstrap required, opened a new log
I20260811 02:50:59.439755 22319 raft_consensus.cc:348] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adec2c93cb1643acb4b917e36fd27ec5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 45439 } }
I20260811 02:50:59.439957 22319 raft_consensus.cc:374] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:50:59.440038 22319 raft_consensus.cc:729] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adec2c93cb1643acb4b917e36fd27ec5, State: Initialized, Role: FOLLOWER
I20260811 02:50:59.440439 22319 consensus_queue.cc:260] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adec2c93cb1643acb4b917e36fd27ec5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 45439 } }
I20260811 02:50:59.440557 22319 raft_consensus.cc:388] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:50:59.440642 22319 raft_consensus.cc:482] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:50:59.440754 22319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:50:59.441774 22319 raft_consensus.cc:504] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adec2c93cb1643acb4b917e36fd27ec5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 45439 } }
I20260811 02:50:59.442063 22319 leader_election.cc:302] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adec2c93cb1643acb4b917e36fd27ec5; no voters:
I20260811 02:50:59.442315 22319 leader_election.cc:288] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:50:59.442556 22328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:50:59.442821 22328 raft_consensus.cc:686] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [term 1 LEADER]: Becoming Leader. State: Replica: adec2c93cb1643acb4b917e36fd27ec5, State: Running, Role: LEADER
I20260811 02:50:59.443183 22328 consensus_queue.cc:237] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adec2c93cb1643acb4b917e36fd27ec5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 45439 } }
I20260811 02:50:59.443671 22319 sys_catalog.cc:564] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:50:59.445031 22330 sys_catalog.cc:455] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adec2c93cb1643acb4b917e36fd27ec5. Latest consensus state: current_term: 1 leader_uuid: "adec2c93cb1643acb4b917e36fd27ec5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adec2c93cb1643acb4b917e36fd27ec5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 45439 } } }
I20260811 02:50:59.445031 22329 sys_catalog.cc:455] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adec2c93cb1643acb4b917e36fd27ec5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adec2c93cb1643acb4b917e36fd27ec5" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 45439 } } }
I20260811 02:50:59.445223 22330 sys_catalog.cc:458] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:50:59.445223 22329 sys_catalog.cc:458] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5 [sys.catalog]: This master's current role is: LEADER
I20260811 02:50:59.445711 22347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:50:59.448871 22347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:50:59.454532 22347 catalog_manager.cc:1347] Generated new cluster ID: ce6f2719125f4995ae35a8761b796214
I20260811 02:50:59.454607 22347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:50:59.476572 22347 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:50:59.477540 22347 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:50:59.488855 22347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5: Generated new TSK 0
I20260811 02:50:59.489600 22347 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:50:59.499660 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:45439
--builtin_ntp_servers=127.21.236.148:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 02:50:59.615274 22358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:59.615520 22358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:59.615581 22358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:59.619540 22358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:59.619591 22358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:59.619679 22358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:50:59.624338 22358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:45439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:59.625380 22358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:59.626819 22358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:59.636129 22368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:59.636137 22370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:59.636404 22358 server_base.cc:1034] running on GCE node
W20260811 02:50:59.636508 22372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:59.636979 22358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:59.637786 22358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:59.638962 22358 hybrid_clock.cc:648] HybridClock initialized: now 1786416659638935 us; error 39 us; skew 500 ppm
I20260811 02:50:59.640844 22358 webserver.cc:460] Webserver started at http://127.21.236.129:39719/ using document root <none> and password file <none>
I20260811 02:50:59.641451 22358 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:59.641548 22358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:59.641782 22358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:59.643862 22358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "12b28cb04cd94c6fb68e2e5e5e152d32"
format_stamp: "Formatted at 2026-08-11 02:50:59 on dist-test-slave-n326"
server_key: "3a3d91c534c7bb841d167470d878f40c"
server_key_iv: "df5644d352596def61e30bada923d9dd"
server_key_version: "encryptionkey@0"
I20260811 02:50:59.644505 22358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "12b28cb04cd94c6fb68e2e5e5e152d32"
format_stamp: "Formatted at 2026-08-11 02:50:59 on dist-test-slave-n326"
server_key: "3a3d91c534c7bb841d167470d878f40c"
server_key_iv: "df5644d352596def61e30bada923d9dd"
server_key_version: "encryptionkey@0"
I20260811 02:50:59.648388 22358 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:50:59.650817 22379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:59.652141 22358 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:50:59.652281 22358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "12b28cb04cd94c6fb68e2e5e5e152d32"
format_stamp: "Formatted at 2026-08-11 02:50:59 on dist-test-slave-n326"
server_key: "3a3d91c534c7bb841d167470d878f40c"
server_key_iv: "df5644d352596def61e30bada923d9dd"
server_key_version: "encryptionkey@0"
I20260811 02:50:59.652390 22358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:59.679313 22358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:59.680016 22358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:59.680222 22358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:59.680833 22358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:59.681818 22358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:50:59.681891 22358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:59.681959 22358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:59.682013 22358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:59.691886 22358 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:40719
I20260811 02:50:59.691915 22564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:40719 every 8 connection(s)
I20260811 02:50:59.692788 22358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:50:59.697341 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 22358
I20260811 02:50:59.697480 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:50:59.697757 22450 external_mini_cluster.cc:1428] Setting key 1017bbef1eed91ae373c5e5af252de26
I20260811 02:50:59.699740 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:45439
--builtin_ntp_servers=127.21.236.148:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 02:50:59.702615 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:59.694685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:52023 (local address 127.21.236.190:45439)
0811 02:50:59.694850 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:59.694856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:59.696013 (+ 1157us) server_negotiation.cc:407] Connection header received
0811 02:50:59.697524 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:59.697527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:59.697586 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:59.697695 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:59.698772 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.699379 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.700595 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.700828 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.700950 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:59.702078 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:59.702081 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:59.702084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:50:59.702105 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:59.702124 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:50:59.702132 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:59.702212 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:59.702402 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 02:50:59.702472 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:50:59.703523 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:59.695158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:45439 (local address 127.21.236.129:52023)
0811 02:50:59.695865 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:59.695909 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:59.697284 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:59.697904 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:59.697913 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:59.698004 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:59.698621 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:59.698638 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.699516 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.699520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:59.700455 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:59.700470 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.700988 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.700992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:59.701159 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:59.701813 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:50:59.701815 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:59.701855 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:50:59.701920 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:59.702227 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:59.702240 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:59.702250 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 02:50:59.702398 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 02:50:59.702679 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":115,"threads_started":1}
I20260811 02:50:59.704769 22565 heartbeater.cc:347] Connected to a master server at 127.21.236.190:45439
I20260811 02:50:59.705039 22565 heartbeater.cc:464] Registering TS with master...
I20260811 02:50:59.705551 22565 heartbeater.cc:511] Master 127.21.236.190:45439 requested a full tablet report, sending...
I20260811 02:50:59.707271 22262 ts_manager.cc:194] Registered new tserver with Master: 12b28cb04cd94c6fb68e2e5e5e152d32 (127.21.236.129:40719)
I20260811 02:50:59.709012 22262 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.236.129:52023
W20260811 02:50:59.816926 22568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:50:59.817196 22568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:50:59.817234 22568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:50:59.821233 22568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:50:59.821293 22568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:50:59.821347 22568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:50:59.826093 22568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:45439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:50:59.827291 22568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:50:59.828667 22568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:50:59.838593 22581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:59.838593 22579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:50:59.838634 22578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:50:59.839042 22568 server_base.cc:1034] running on GCE node
I20260811 02:50:59.839546 22568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:50:59.840215 22568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:50:59.841466 22568 hybrid_clock.cc:648] HybridClock initialized: now 1786416659841450 us; error 29 us; skew 500 ppm
I20260811 02:50:59.843622 22568 webserver.cc:460] Webserver started at http://127.21.236.130:40865/ using document root <none> and password file <none>
I20260811 02:50:59.844281 22568 fs_manager.cc:359] Metadata directory not provided
I20260811 02:50:59.844422 22568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:50:59.844743 22568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:50:59.847237 22568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "aeff4fd97be74008bc5052e7f36a838a"
format_stamp: "Formatted at 2026-08-11 02:50:59 on dist-test-slave-n326"
server_key: "a427f493c0246f5ad6774d35a2da9767"
server_key_iv: "8251c73a499fabbfb19a07638bd9d65b"
server_key_version: "encryptionkey@0"
I20260811 02:50:59.847898 22568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "aeff4fd97be74008bc5052e7f36a838a"
format_stamp: "Formatted at 2026-08-11 02:50:59 on dist-test-slave-n326"
server_key: "a427f493c0246f5ad6774d35a2da9767"
server_key_iv: "8251c73a499fabbfb19a07638bd9d65b"
server_key_version: "encryptionkey@0"
I20260811 02:50:59.852054 22568 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:50:59.854722 22594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:59.856365 22568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:50:59.856633 22568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "aeff4fd97be74008bc5052e7f36a838a"
format_stamp: "Formatted at 2026-08-11 02:50:59 on dist-test-slave-n326"
server_key: "a427f493c0246f5ad6774d35a2da9767"
server_key_iv: "8251c73a499fabbfb19a07638bd9d65b"
server_key_version: "encryptionkey@0"
I20260811 02:50:59.856810 22568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:50:59.868827 22568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:50:59.869565 22568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:50:59.869783 22568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:50:59.870453 22568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:50:59.871570 22568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:50:59.871625 22568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:59.871670 22568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:50:59.871733 22568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:50:59.882002 22568 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:34345
I20260811 02:50:59.882035 22771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:34345 every 8 connection(s)
I20260811 02:50:59.883000 22568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:50:59.887759 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 22568
I20260811 02:50:59.887909 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:50:59.888259 22450 external_mini_cluster.cc:1428] Setting key 8e0ddeb9ea0e4570fc5d671f88f0bd4d
I20260811 02:50:59.890638 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:45439
--builtin_ntp_servers=127.21.236.148:32897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 02:50:59.894105 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:59.885158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:35967 (local address 127.21.236.190:45439)
0811 02:50:59.885296 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 02:50:59.885305 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:50:59.886326 (+ 1021us) server_negotiation.cc:407] Connection header received
0811 02:50:59.888018 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:50:59.888021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:50:59.888081 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:50:59.888157 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:50:59.889296 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.889951 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.891737 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.891956 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.892037 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:59.893483 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:50:59.893489 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:50:59.893495 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:50:59.893512 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:50:59.893540 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:50:59.893556 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:50:59.893655 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 02:50:59.893799 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 02:50:59.893892 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:50:59.894855 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:50:59.885494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:45439 (local address 127.21.236.130:35967)
0811 02:50:59.886175 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0811 02:50:59.886215 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 02:50:59.887765 (+ 1550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:50:59.888360 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:50:59.888369 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:50:59.888476 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:50:59.889135 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:59.889154 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.890076 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.890079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:59.891610 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:50:59.891625 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:50:59.892118 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:50:59.892125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:50:59.892362 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:50:59.893200 (+ 838us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:50:59.893202 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:50:59.893244 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:50:59.893309 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:50:59.893654 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:50:59.893661 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:50:59.893665 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 02:50:59.893761 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 02:50:59.893981 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":129,"threads_started":1}
I20260811 02:50:59.896131 22773 heartbeater.cc:347] Connected to a master server at 127.21.236.190:45439
I20260811 02:50:59.896427 22773 heartbeater.cc:464] Registering TS with master...
I20260811 02:50:59.897197 22773 heartbeater.cc:511] Master 127.21.236.190:45439 requested a full tablet report, sending...
I20260811 02:50:59.898437 22262 ts_manager.cc:194] Registered new tserver with Master: aeff4fd97be74008bc5052e7f36a838a (127.21.236.130:34345)
I20260811 02:50:59.899020 22262 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.236.130:35967
W20260811 02:51:00.012483 22776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:00.012759 22776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:00.012799 22776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:00.016908 22776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:00.016966 22776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:00.017055 22776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:51:00.021996 22776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:32897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:45439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:00.023248 22776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:00.024699 22776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:00.034854 22785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:00.034970 22776 server_base.cc:1034] running on GCE node
W20260811 02:51:00.034854 22787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:00.035261 22783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:00.035701 22776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:00.036316 22776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:00.037503 22776 hybrid_clock.cc:648] HybridClock initialized: now 1786416660037487 us; error 29 us; skew 500 ppm
I20260811 02:51:00.039517 22776 webserver.cc:460] Webserver started at http://127.21.236.131:35429/ using document root <none> and password file <none>
I20260811 02:51:00.040105 22776 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:00.040169 22776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:00.040387 22776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:00.042474 22776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "5c64926ebd1c4d08a9866482ce6a6e91"
format_stamp: "Formatted at 2026-08-11 02:51:00 on dist-test-slave-n326"
server_key: "54afc926cc37128cea2f6a173987d7cc"
server_key_iv: "d4d8541b4893df351cbc34f22b8f8f42"
server_key_version: "encryptionkey@0"
I20260811 02:51:00.043035 22776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "5c64926ebd1c4d08a9866482ce6a6e91"
format_stamp: "Formatted at 2026-08-11 02:51:00 on dist-test-slave-n326"
server_key: "54afc926cc37128cea2f6a173987d7cc"
server_key_iv: "d4d8541b4893df351cbc34f22b8f8f42"
server_key_version: "encryptionkey@0"
I20260811 02:51:00.047015 22776 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:51:00.049574 22800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:00.050896 22776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:51:00.051021 22776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "5c64926ebd1c4d08a9866482ce6a6e91"
format_stamp: "Formatted at 2026-08-11 02:51:00 on dist-test-slave-n326"
server_key: "54afc926cc37128cea2f6a173987d7cc"
server_key_iv: "d4d8541b4893df351cbc34f22b8f8f42"
server_key_version: "encryptionkey@0"
I20260811 02:51:00.051180 22776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:00.062014 22776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:00.062660 22776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:00.062851 22776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:00.063534 22776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:51:00.064464 22776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:00.064509 22776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:00.064576 22776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:00.064611 22776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:00.073652 22776 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:43453
I20260811 02:51:00.073675 22988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:43453 every 8 connection(s)
I20260811 02:51:00.074553 22776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:51:00.079353 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 22776
I20260811 02:51:00.079497 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:51:00.079766 22450 external_mini_cluster.cc:1428] Setting key 7e85e30ce61d38a6c005403d13adfde6
I20260811 02:51:00.083938 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.076326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:57527 (local address 127.21.236.190:45439)
0811 02:51:00.076530 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:00.076535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:00.077497 (+ 962us) server_negotiation.cc:407] Connection header received
0811 02:51:00.078857 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:00.078860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:00.078920 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:00.078997 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:00.080022 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.080670 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.081804 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.082103 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.082218 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:00.083420 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:00.083424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:00.083427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:00.083440 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:00.083470 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:00.083480 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:00.083557 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:00.083727 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 02:51:00.083790 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 02:51:00.084761 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.076734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:45439 (local address 127.21.236.131:57527)
0811 02:51:00.077359 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:00.077397 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:00.078652 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:00.079202 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:00.079209 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:00.079290 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:00.079858 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:00.079879 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.080795 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.080799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:00.081673 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:00.081682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.082256 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.082260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:00.082433 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:00.083113 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:51:00.083115 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:00.083183 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:51:00.083270 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:00.083572 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:00.083583 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:00.083589 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 02:51:00.083722 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 02:51:00.084007 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
I20260811 02:51:00.085807 22989 heartbeater.cc:347] Connected to a master server at 127.21.236.190:45439
I20260811 02:51:00.086058 22989 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:00.086560 22989 heartbeater.cc:511] Master 127.21.236.190:45439 requested a full tablet report, sending...
I20260811 02:51:00.087602 22262 ts_manager.cc:194] Registered new tserver with Master: 5c64926ebd1c4d08a9866482ce6a6e91 (127.21.236.131:43453)
I20260811 02:51:00.088171 22262 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.236.131:57527
I20260811 02:51:00.093568 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:51:00.099442 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.095608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39200 (local address 127.21.236.190:45439)
0811 02:51:00.095721 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:00.095725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:00.095941 (+ 216us) server_negotiation.cc:407] Connection header received
0811 02:51:00.096008 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:00.096010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:00.096050 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:00.096121 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:00.096398 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.096895 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.097690 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.097904 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.097986 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:00.098792 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:00.098795 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:00.098797 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:00.098809 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:00.098829 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:00.098837 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:00.098968 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:00.099199 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 02:51:00.099247 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 02:51:00.102581 22262 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:51:00.105855 22262 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:51:00.120579 23004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.114716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:34345 (local address 127.0.0.1:41122)
0811 02:51:00.115536 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:00.115590 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:00.115729 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:00.116478 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:00.116480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:00.116515 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:00.116722 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:00.116728 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.117765 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.117770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:00.118816 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:00.118826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.119986 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.119990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:00.120108 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:00.120155 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 02:51:00.120270 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 02:51:00.120391 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"mutex_wait_us":53,"spinlock_wait_cycles":8448,"thread_start_us":107,"threads_started":1}
I20260811 02:51:00.120586 23009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.115340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41122 (local address 127.21.236.130:34345)
0811 02:51:00.116006 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:00.116017 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:00.116041 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:51:00.116123 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:00.116128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:00.116326 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:00.116613 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:00.116860 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.117645 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.118975 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.119823 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.120007 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:00.120200 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:00.120297 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 02:51:00.120395 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":139,"threads_started":1}
I20260811 02:51:00.121352 23002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.114935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:40719 (local address 127.0.0.1:34690)
0811 02:51:00.115962 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:00.115974 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:00.116051 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:00.116708 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:00.116711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:00.116726 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:00.116821 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:00.116827 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.118530 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.118535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:00.119766 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:00.119777 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.120943 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.120946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:00.121032 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:00.121056 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:51:00.121121 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 02:51:00.121194 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":24,"thread_start_us":98,"threads_started":1}
I20260811 02:51:00.121541 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.115353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34690 (local address 127.21.236.129:40719)
0811 02:51:00.115804 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:00.115815 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:00.116142 (+ 327us) server_negotiation.cc:407] Connection header received
0811 02:51:00.116220 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:00.116226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:00.116395 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:00.116537 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:00.117016 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.118317 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.119918 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.120801 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.120945 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:00.121155 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:00.121221 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 02:51:00.121351 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":137,"threads_started":1}
I20260811 02:51:00.121773 23008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.114514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:43453 (local address 127.0.0.1:46990)
0811 02:51:00.115536 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:00.115590 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:00.115729 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:00.116473 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:00.116479 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:00.116517 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:00.116705 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:00.116714 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.118557 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.118560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:00.119854 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:00.119868 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.121353 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.121371 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:00.121497 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:00.121519 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:51:00.121581 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 02:51:00.121648 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"mutex_wait_us":118,"spinlock_wait_cycles":12160,"thread_start_us":96,"threads_started":1}
I20260811 02:51:00.122426 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.114717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46990 (local address 127.21.236.131:43453)
0811 02:51:00.115370 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:00.115382 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:00.115874 (+ 492us) server_negotiation.cc:407] Connection header received
0811 02:51:00.115958 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:00.115963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:00.116182 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:00.116325 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:00.116820 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.118290 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.119997 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.120806 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.120939 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:00.122086 (+ 1147us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:00.122151 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 02:51:00.122252 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":182,"threads_started":1}
I20260811 02:51:00.124810 22875 tablet_service.cc:1467] Processing CreateTablet for tablet 9271719d821d4565aabf679290772409 (DEFAULT_TABLE table=test-table [id=91ed993054c94db8987c7c20fa3af6c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:00.124671 22668 tablet_service.cc:1467] Processing CreateTablet for tablet 9271719d821d4565aabf679290772409 (DEFAULT_TABLE table=test-table [id=91ed993054c94db8987c7c20fa3af6c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:00.125175 22466 tablet_service.cc:1467] Processing CreateTablet for tablet 9271719d821d4565aabf679290772409 (DEFAULT_TABLE table=test-table [id=91ed993054c94db8987c7c20fa3af6c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:00.125852 22875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9271719d821d4565aabf679290772409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:00.125861 22668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9271719d821d4565aabf679290772409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:00.126191 22466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9271719d821d4565aabf679290772409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:00.132040 23011 tablet_bootstrap.cc:492] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91: Bootstrap starting.
I20260811 02:51:00.133556 23015 tablet_bootstrap.cc:492] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a: Bootstrap starting.
I20260811 02:51:00.134341 23012 tablet_bootstrap.cc:492] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32: Bootstrap starting.
I20260811 02:51:00.134552 23011 tablet_bootstrap.cc:654] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:00.135492 23011 log.cc:824] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:00.136479 23012 tablet_bootstrap.cc:654] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:00.136695 23015 tablet_bootstrap.cc:654] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:00.137296 23012 log.cc:824] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:00.137423 23015 log.cc:824] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:00.137713 23011 tablet_bootstrap.cc:492] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91: No bootstrap required, opened a new log
I20260811 02:51:00.137915 23011 ts_tablet_manager.cc:1397] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:51:00.139271 23015 tablet_bootstrap.cc:492] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a: No bootstrap required, opened a new log
I20260811 02:51:00.139426 23012 tablet_bootstrap.cc:492] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32: No bootstrap required, opened a new log
I20260811 02:51:00.139504 23015 ts_tablet_manager.cc:1397] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 02:51:00.139607 23012 ts_tablet_manager.cc:1397] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260811 02:51:00.142513 23011 raft_consensus.cc:348] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43453 } } peers { permanent_uuid: "aeff4fd97be74008bc5052e7f36a838a" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 34345 } } peers { permanent_uuid: "12b28cb04cd94c6fb68e2e5e5e152d32" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40719 } }
I20260811 02:51:00.142798 23011 raft_consensus.cc:374] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:00.142856 23011 raft_consensus.cc:729] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c64926ebd1c4d08a9866482ce6a6e91, State: Initialized, Role: FOLLOWER
I20260811 02:51:00.143038 23015 raft_consensus.cc:348] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43453 } } peers { permanent_uuid: "aeff4fd97be74008bc5052e7f36a838a" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 34345 } } peers { permanent_uuid: "12b28cb04cd94c6fb68e2e5e5e152d32" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40719 } }
I20260811 02:51:00.143337 23015 raft_consensus.cc:374] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:00.143410 23015 raft_consensus.cc:729] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeff4fd97be74008bc5052e7f36a838a, State: Initialized, Role: FOLLOWER
I20260811 02:51:00.143527 23011 consensus_queue.cc:260] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43453 } } peers { permanent_uuid: "aeff4fd97be74008bc5052e7f36a838a" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 34345 } } peers { permanent_uuid: "12b28cb04cd94c6fb68e2e5e5e152d32" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40719 } }
I20260811 02:51:00.143862 23015 consensus_queue.cc:260] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43453 } } peers { permanent_uuid: "aeff4fd97be74008bc5052e7f36a838a" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 34345 } } peers { permanent_uuid: "12b28cb04cd94c6fb68e2e5e5e152d32" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40719 } }
I20260811 02:51:00.143832 23012 raft_consensus.cc:348] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43453 } } peers { permanent_uuid: "aeff4fd97be74008bc5052e7f36a838a" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 34345 } } peers { permanent_uuid: "12b28cb04cd94c6fb68e2e5e5e152d32" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40719 } }
I20260811 02:51:00.144083 23012 raft_consensus.cc:374] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:00.144174 23012 raft_consensus.cc:729] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12b28cb04cd94c6fb68e2e5e5e152d32, State: Initialized, Role: FOLLOWER
I20260811 02:51:00.144481 22989 heartbeater.cc:503] Master 127.21.236.190:45439 was elected leader, sending a full tablet report...
I20260811 02:51:00.144649 23012 consensus_queue.cc:260] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43453 } } peers { permanent_uuid: "aeff4fd97be74008bc5052e7f36a838a" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 34345 } } peers { permanent_uuid: "12b28cb04cd94c6fb68e2e5e5e152d32" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40719 } }
I20260811 02:51:00.144760 23011 ts_tablet_manager.cc:1428] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260811 02:51:00.144857 23015 ts_tablet_manager.cc:1428] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260811 02:51:00.145167 22773 heartbeater.cc:503] Master 127.21.236.190:45439 was elected leader, sending a full tablet report...
I20260811 02:51:00.145275 22565 heartbeater.cc:503] Master 127.21.236.190:45439 was elected leader, sending a full tablet report...
I20260811 02:51:00.145609 23012 ts_tablet_manager.cc:1428] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260811 02:51:00.194013 22566 tablet.cc:2369] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:51:00.225062 23020 raft_consensus.cc:482] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:51:00.225340 23020 raft_consensus.cc:504] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43453 } } peers { permanent_uuid: "aeff4fd97be74008bc5052e7f36a838a" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 34345 } } peers { permanent_uuid: "12b28cb04cd94c6fb68e2e5e5e152d32" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40719 } }
I20260811 02:51:00.226718 23020 leader_election.cc:288] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aeff4fd97be74008bc5052e7f36a838a (127.21.236.130:34345), 12b28cb04cd94c6fb68e2e5e5e152d32 (127.21.236.129:40719)
I20260811 02:51:00.230774 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.226923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:34345 (local address 127.21.236.131:59057)
0811 02:51:00.227115 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:00.227131 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:00.227233 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:00.227577 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:00.227580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:00.227647 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:00.227803 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:00.227811 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.228698 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.228701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:00.229580 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:00.229590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.230291 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.230295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:00.230450 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:00.230478 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 02:51:00.230547 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 02:51:00.230633 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"spinlock_wait_cycles":13952}
I20260811 02:51:00.230803 23009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.227177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:59057 (local address 127.21.236.130:34345)
0811 02:51:00.227317 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:00.227324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:00.227340 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:51:00.227400 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:00.227403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:00.227453 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:00.227551 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:00.227968 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.228579 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.229719 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.230173 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.230270 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:00.230311 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:00.230624 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 02:51:00.230696 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:51:00.230984 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.227180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:41041 (local address 127.21.236.129:40719)
0811 02:51:00.227318 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:00.227323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:00.227633 (+ 310us) server_negotiation.cc:407] Connection header received
0811 02:51:00.227784 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:00.227787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:00.227840 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:00.227954 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:00.228256 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.228826 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.229899 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.230363 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.230485 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:00.230553 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:00.230782 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 02:51:00.230859 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 02:51:00.231356 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.227093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:40719 (local address 127.21.236.131:41041)
0811 02:51:00.227523 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:00.227539 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:00.227662 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:00.227973 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:00.227976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:00.227994 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:00.228131 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:00.228137 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.228959 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.228962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:00.229764 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:00.229774 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.230509 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.230516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:00.230628 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:00.230650 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 02:51:00.231164 (+ 514us) client_negotiation.cc:240] Negotiation successful
0811 02:51:00.231232 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"spinlock_wait_cycles":10624,"thread_start_us":117,"threads_started":1}
I20260811 02:51:00.231681 22697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9271719d821d4565aabf679290772409" candidate_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeff4fd97be74008bc5052e7f36a838a" is_pre_election: true
I20260811 02:51:00.232002 22697 raft_consensus.cc:2454] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c64926ebd1c4d08a9866482ce6a6e91 in term 0.
I20260811 02:51:00.232007 22498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9271719d821d4565aabf679290772409" candidate_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12b28cb04cd94c6fb68e2e5e5e152d32" is_pre_election: true
I20260811 02:51:00.232301 22498 raft_consensus.cc:2454] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c64926ebd1c4d08a9866482ce6a6e91 in term 0.
I20260811 02:51:00.232571 22801 leader_election.cc:302] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c64926ebd1c4d08a9866482ce6a6e91, aeff4fd97be74008bc5052e7f36a838a; no voters:
I20260811 02:51:00.232854 23020 raft_consensus.cc:2781] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:51:00.232944 23020 raft_consensus.cc:482] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:51:00.232987 23020 raft_consensus.cc:3037] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:00.234088 23020 raft_consensus.cc:504] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43453 } } peers { permanent_uuid: "aeff4fd97be74008bc5052e7f36a838a" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 34345 } } peers { permanent_uuid: "12b28cb04cd94c6fb68e2e5e5e152d32" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40719 } }
I20260811 02:51:00.234458 23020 leader_election.cc:288] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [CANDIDATE]: Term 1 election: Requested vote from peers aeff4fd97be74008bc5052e7f36a838a (127.21.236.130:34345), 12b28cb04cd94c6fb68e2e5e5e152d32 (127.21.236.129:40719)
I20260811 02:51:00.234787 22697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9271719d821d4565aabf679290772409" candidate_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeff4fd97be74008bc5052e7f36a838a"
I20260811 02:51:00.234920 22697 raft_consensus.cc:3037] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:00.234967 22498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9271719d821d4565aabf679290772409" candidate_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12b28cb04cd94c6fb68e2e5e5e152d32"
I20260811 02:51:00.235147 22498 raft_consensus.cc:3037] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:00.236042 22697 raft_consensus.cc:2454] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c64926ebd1c4d08a9866482ce6a6e91 in term 1.
I20260811 02:51:00.236280 22498 raft_consensus.cc:2454] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c64926ebd1c4d08a9866482ce6a6e91 in term 1.
I20260811 02:51:00.236392 22801 leader_election.cc:302] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c64926ebd1c4d08a9866482ce6a6e91, aeff4fd97be74008bc5052e7f36a838a; no voters:
I20260811 02:51:00.236624 23020 raft_consensus.cc:2781] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:00.236886 23020 raft_consensus.cc:686] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [term 1 LEADER]: Becoming Leader. State: Replica: 5c64926ebd1c4d08a9866482ce6a6e91, State: Running, Role: LEADER
I20260811 02:51:00.237205 23020 consensus_queue.cc:237] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43453 } } peers { permanent_uuid: "aeff4fd97be74008bc5052e7f36a838a" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 34345 } } peers { permanent_uuid: "12b28cb04cd94c6fb68e2e5e5e152d32" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40719 } }
I20260811 02:51:00.240513 22261 catalog_manager.cc:5495] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c64926ebd1c4d08a9866482ce6a6e91 (127.21.236.131). New cstate: current_term: 1 leader_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c64926ebd1c4d08a9866482ce6a6e91" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 43453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aeff4fd97be74008bc5052e7f36a838a" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 34345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12b28cb04cd94c6fb68e2e5e5e152d32" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40719 } health_report { overall_health: UNKNOWN } } }
I20260811 02:51:00.280194 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.276422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47004 (local address 127.21.236.131:43453)
0811 02:51:00.276686 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:00.276693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:00.276710 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:51:00.276801 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:00.276803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:00.276860 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:00.276977 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:00.277250 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.277761 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.278641 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:00.278862 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:00.278977 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:00.279331 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:00.279749 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:00.279920 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:00.280006 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 02:51:00.280040 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260811 02:51:00.286496 22697 raft_consensus.cc:1261] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a [term 1 FOLLOWER]: Refusing update from remote peer 5c64926ebd1c4d08a9866482ce6a6e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:00.286526 22498 raft_consensus.cc:1261] T 9271719d821d4565aabf679290772409 P 12b28cb04cd94c6fb68e2e5e5e152d32 [term 1 FOLLOWER]: Refusing update from remote peer 5c64926ebd1c4d08a9866482ce6a6e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:00.287286 23020 consensus_queue.cc:1035] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12b28cb04cd94c6fb68e2e5e5e152d32" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 40719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:00.287482 23024 consensus_queue.cc:1035] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeff4fd97be74008bc5052e7f36a838a" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 34345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:00.296205 23029 mvcc.cc:205] Tried to move back new op lower bound from 7317162640525361152 to 7317162640334094336. Current Snapshot: MvccSnapshot[applied={T|T < 7317162640525361152}]
I20260811 02:51:00.296305 23033 mvcc.cc:205] Tried to move back new op lower bound from 7317162640525361152 to 7317162640334094336. Current Snapshot: MvccSnapshot[applied={T|T < 7317162640525361152}]
I20260811 02:51:00.296388 23031 mvcc.cc:205] Tried to move back new op lower bound from 7317162640525361152 to 7317162640334094336. Current Snapshot: MvccSnapshot[applied={T|T < 7317162640525361152}]
I20260811 02:51:00.307914 22260 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:51:00.308122 22260 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:51:00.310814 22260 catalog_manager.cc:5782] T 00000000000000000000000000000000 P adec2c93cb1643acb4b917e36fd27ec5: Sending DeleteTablet for 3 replicas of tablet 9271719d821d4565aabf679290772409
I20260811 02:51:00.311745 22875 tablet_service.cc:1514] Processing DeleteTablet for tablet 9271719d821d4565aabf679290772409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:00 UTC) from {username='slave'} at 127.0.0.1:46990
I20260811 02:51:00.311770 22466 tablet_service.cc:1514] Processing DeleteTablet for tablet 9271719d821d4565aabf679290772409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:00 UTC) from {username='slave'} at 127.0.0.1:34690
I20260811 02:51:00.312338 22668 tablet_service.cc:1514] Processing DeleteTablet for tablet 9271719d821d4565aabf679290772409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:00 UTC) from {username='slave'} at 127.0.0.1:41122
I20260811 02:51:00.312497 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 22358
I20260811 02:51:00.312877 23046 tablet_replica.cc:330] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91: stopping tablet replica
I20260811 02:51:00.313264 23046 raft_consensus.cc:2229] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:51:00.313637 23046 raft_consensus.cc:2258] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:51:00.319306 23046 ts_tablet_manager.cc:1905] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:00.322392 23046 ts_tablet_manager.cc:1918] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:51:00.322520 23046 log.cc:1196] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/9271719d821d4565aabf679290772409
I20260811 02:51:00.322854 23046 ts_tablet_manager.cc:1939] T 9271719d821d4565aabf679290772409 P 5c64926ebd1c4d08a9866482ce6a6e91: Deleting consensus metadata
I20260811 02:51:00.324328 23047 tablet_replica.cc:330] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a: stopping tablet replica
I20260811 02:51:00.324328 22244 catalog_manager.cc:4841] TS 5c64926ebd1c4d08a9866482ce6a6e91 (127.21.236.131:43453): tablet 9271719d821d4565aabf679290772409 (table test-table [id=91ed993054c94db8987c7c20fa3af6c0]) successfully deleted
I20260811 02:51:00.324712 23047 raft_consensus.cc:2229] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:51:00.325064 23047 raft_consensus.cc:2258] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:51:00.326707 23047 ts_tablet_manager.cc:1905] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:00.329528 23047 ts_tablet_manager.cc:1918] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:51:00.329640 23047 log.cc:1196] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/9271719d821d4565aabf679290772409
I20260811 02:51:00.329943 23047 ts_tablet_manager.cc:1939] T 9271719d821d4565aabf679290772409 P aeff4fd97be74008bc5052e7f36a838a: Deleting consensus metadata
W20260811 02:51:00.330334 22240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:40719 (error 108)
I20260811 02:51:00.331305 22239 catalog_manager.cc:4841] TS aeff4fd97be74008bc5052e7f36a838a (127.21.236.130:34345): tablet 9271719d821d4565aabf679290772409 (table test-table [id=91ed993054c94db8987c7c20fa3af6c0]) successfully deleted
I20260811 02:51:00.331580 23008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:00.331213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:40719 (local address 127.0.0.1:34700)
0811 02:51:00.331379 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:00.331486 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:40719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 02:51:00.331710 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 22568
W20260811 02:51:00.347491 22240 catalog_manager.cc:4584] TS 12b28cb04cd94c6fb68e2e5e5e152d32 (127.21.236.129:40719): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9271719d821d4565aabf679290772409: Network error: Client connection negotiation failed: client connection to 127.21.236.129:40719: connect: Connection refused (error 111)
I20260811 02:51:00.355235 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 22776
I20260811 02:51:00.368940 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 22215
2026-08-11T02:51:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1173 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:00 dist-test-slave-n326 krb5kdc[23055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:00 dist-test-slave-n326 krb5kdc[23055](info): set up 2 sockets
Aug 11 02:51:00 dist-test-slave-n326 krb5kdc[23055](info): commencing operation
krb5kdc: starting...
W20260811 02:51:01.741847 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.342s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:01 dist-test-slave-n326 krb5kdc[23055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:51:01.871708 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:39493
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:39493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:01.982832 23076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:01.983176 23076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:01.983265 23076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:01.987214 23076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:01.987303 23076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:01.987342 23076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:01.987380 23076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:01.987411 23076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:01.992589 23076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:39493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:39493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:01.993665 23076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:01.994683 23076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:02.003369 23085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:02.003420 23089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:02.003402 23087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:02.003489 23076 server_base.cc:1034] running on GCE node
I20260811 02:51:02.004287 23076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:02.005456 23076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:02.006654 23076 hybrid_clock.cc:648] HybridClock initialized: now 1786416662006633 us; error 35 us; skew 500 ppm
Aug 11 02:51:02 dist-test-slave-n326 krb5kdc[23055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:02.009817 23076 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:51:02.010795 23076 webserver.cc:460] Webserver started at http://127.21.236.190:34331/ using document root <none> and password file <none>
I20260811 02:51:02.011485 23076 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:02.011597 23076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:02.011858 23076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:02.014096 23076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "b5169c3619c24a9696c4d3ed0a7e9a79"
format_stamp: "Formatted at 2026-08-11 02:51:02 on dist-test-slave-n326"
server_key: "822400237083a840bfafeaef456c0306"
server_key_iv: "ddd58237f11add267f22c09602caac02"
server_key_version: "encryptionkey@0"
I20260811 02:51:02.014721 23076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "b5169c3619c24a9696c4d3ed0a7e9a79"
format_stamp: "Formatted at 2026-08-11 02:51:02 on dist-test-slave-n326"
server_key: "822400237083a840bfafeaef456c0306"
server_key_iv: "ddd58237f11add267f22c09602caac02"
server_key_version: "encryptionkey@0"
I20260811 02:51:02.018858 23076 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 02:51:02.021471 23104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:02.022754 23076 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:51:02.022917 23076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "b5169c3619c24a9696c4d3ed0a7e9a79"
format_stamp: "Formatted at 2026-08-11 02:51:02 on dist-test-slave-n326"
server_key: "822400237083a840bfafeaef456c0306"
server_key_iv: "ddd58237f11add267f22c09602caac02"
server_key_version: "encryptionkey@0"
I20260811 02:51:02.023042 23076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:02.039036 23076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:02.042373 23076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:02.042618 23076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:02.050838 23076 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:39493
I20260811 02:51:02.050886 23192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:39493 every 8 connection(s)
I20260811 02:51:02.051950 23076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:02.054759 23193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:02.058606 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 23076
I20260811 02:51:02.058774 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:02.059128 22450 external_mini_cluster.cc:1428] Setting key a80e2a095aa9826a9585c0c56f46292c
I20260811 02:51:02.060739 23193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79: Bootstrap starting.
I20260811 02:51:02.063469 23193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:02.064368 23193 log.cc:824] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:02.066233 23193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79: No bootstrap required, opened a new log
Aug 11 02:51:02 dist-test-slave-n326 krb5kdc[23055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:51:02.070744 23193 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5169c3619c24a9696c4d3ed0a7e9a79" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39493 } }
I20260811 02:51:02.070993 23193 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:02.071112 23193 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5169c3619c24a9696c4d3ed0a7e9a79, State: Initialized, Role: FOLLOWER
I20260811 02:51:02.071661 23193 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5169c3619c24a9696c4d3ed0a7e9a79" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39493 } }
I20260811 02:51:02.071842 23193 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:02.071921 23193 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:02.072041 23193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:02.073166 23193 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5169c3619c24a9696c4d3ed0a7e9a79" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39493 } }
I20260811 02:51:02.073534 23193 leader_election.cc:302] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5169c3619c24a9696c4d3ed0a7e9a79; no voters:
I20260811 02:51:02.073881 23193 leader_election.cc:288] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:02.073992 23200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:02.074201 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:02.060579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53844 (local address 127.21.236.190:39493)
0811 02:51:02.061014 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:02.061032 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:02.061084 (+ 52us) server_negotiation.cc:407] Connection header received
0811 02:51:02.061912 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:02.061940 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:02.062366 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:02.062703 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:02.063506 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.064940 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.065820 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.066287 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.066498 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:02.069230 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:02.069235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:02.069247 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:02.069274 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:02.070971 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:02.071572 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:02.071576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:02.071580 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:02.071689 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:02.072009 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:02.072014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:02.072019 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:02.072466 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:02.072630 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:02.072907 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 02:51:02.073193 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":124,"threads_started":1}
I20260811 02:51:02.075176 23193 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:02.074198 23200 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [term 1 LEADER]: Becoming Leader. State: Replica: b5169c3619c24a9696c4d3ed0a7e9a79, State: Running, Role: LEADER
I20260811 02:51:02.075598 23200 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5169c3619c24a9696c4d3ed0a7e9a79" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39493 } }
I20260811 02:51:02.077270 23201 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5169c3619c24a9696c4d3ed0a7e9a79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5169c3619c24a9696c4d3ed0a7e9a79" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39493 } } }
I20260811 02:51:02.077392 23201 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:02.077659 23204 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5169c3619c24a9696c4d3ed0a7e9a79. Latest consensus state: current_term: 1 leader_uuid: "b5169c3619c24a9696c4d3ed0a7e9a79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5169c3619c24a9696c4d3ed0a7e9a79" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 39493 } } }
I20260811 02:51:02.077876 23204 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:02.078069 23223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:02.081074 23223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:02.086688 23223 catalog_manager.cc:1347] Generated new cluster ID: e926d88f2d2a41c1a77a516a312aab8e
I20260811 02:51:02.086804 23223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:02.101368 23223 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:02.102300 23223 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:02.111338 23223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79: Generated new TSK 0
I20260811 02:51:02.112022 23223 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.236.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
Entry for principal kudu/127.21.236.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab.
I20260811 02:51:02.146631 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39493
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:02.266230 23230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:02.266494 23230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:02.266537 23230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:02.270538 23230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:02.270607 23230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:02.270666 23230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:51:02.276286 23230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.129.keytab
--principal=kudu/127.21.236.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:02.277382 23230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:02.278429 23230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:02.288410 23237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:02.288465 23230 server_base.cc:1034] running on GCE node
W20260811 02:51:02.288475 23238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:02.288410 23244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:02.289094 23230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:02.289793 23230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:02.290990 23230 hybrid_clock.cc:648] HybridClock initialized: now 1786416662290977 us; error 26 us; skew 500 ppm
Aug 11 02:51:02 dist-test-slave-n326 krb5kdc[23055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:02.293856 23230 init.cc:377] Logged in from keytab as kudu/127.21.236.129@KRBTEST.COM (short username kudu)
I20260811 02:51:02.294844 23230 webserver.cc:460] Webserver started at http://127.21.236.129:39533/ using document root <none> and password file <none>
I20260811 02:51:02.295501 23230 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:02.295598 23230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:02.295837 23230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:02.297902 23230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "b06f160bb556447096382c078889312a"
format_stamp: "Formatted at 2026-08-11 02:51:02 on dist-test-slave-n326"
server_key: "52696ce041799a0a9a936b230b9805cc"
server_key_iv: "91056ae102d14aed351fbef642405157"
server_key_version: "encryptionkey@0"
I20260811 02:51:02.298478 23230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "b06f160bb556447096382c078889312a"
format_stamp: "Formatted at 2026-08-11 02:51:02 on dist-test-slave-n326"
server_key: "52696ce041799a0a9a936b230b9805cc"
server_key_iv: "91056ae102d14aed351fbef642405157"
server_key_version: "encryptionkey@0"
I20260811 02:51:02.302470 23230 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:51:02.305122 23256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:02.306507 23230 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:51:02.306676 23230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "b06f160bb556447096382c078889312a"
format_stamp: "Formatted at 2026-08-11 02:51:02 on dist-test-slave-n326"
server_key: "52696ce041799a0a9a936b230b9805cc"
server_key_iv: "91056ae102d14aed351fbef642405157"
server_key_version: "encryptionkey@0"
I20260811 02:51:02.306803 23230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:02.327322 23230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:02.331000 23230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:02.331250 23230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:02.331907 23230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:51:02.332798 23230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:02.332886 23230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:02.332962 23230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:02.333000 23230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:02.342401 23230 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:41787
I20260811 02:51:02.342454 23432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:41787 every 8 connection(s)
I20260811 02:51:02.343393 23230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:51:02.344089 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 23230
I20260811 02:51:02.344234 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:51:02.344585 22450 external_mini_cluster.cc:1428] Setting key 784346ca6b53b020b0b9410921b22fe6
Aug 11 02:51:02 dist-test-slave-n326 krb5kdc[23055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.129@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.236.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.130@KRBTEST.COM" created.
I20260811 02:51:02.359503 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:02.345517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:55299 (local address 127.21.236.190:39493)
0811 02:51:02.345651 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:02.345656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:02.346854 (+ 1198us) server_negotiation.cc:407] Connection header received
0811 02:51:02.348918 (+ 2064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:02.348923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:02.348988 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:02.349090 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:02.350347 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.350926 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.352148 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.352379 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.352486 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:02.355340 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:02.355343 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:02.355345 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:02.355372 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:02.357139 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:02.357698 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:02.357702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:02.357703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:02.357789 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:02.358218 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:02.358221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:02.358223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:02.358421 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:02.358534 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:02.359187 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 02:51:02.359320 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:51:02.360311 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:02.346036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:39493 (local address 127.21.236.129:55299)
0811 02:51:02.346675 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:02.346733 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:02.348644 (+ 1911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:02.349317 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:02.349324 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:02.349705 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:02.350191 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:02.350205 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.351085 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.351089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:02.352003 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:02.352014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.352542 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.352547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:02.352719 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:02.353336 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:02.353353 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:02.355164 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:02.357278 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:02.357284 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:02.357299 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:02.357582 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:02.357895 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:02.357932 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:02.357934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:02.358081 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:02.358532 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:02.358539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:02.358860 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 02:51:02.359105 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 02:51:02.359464 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":136,"threads_started":1}
I20260811 02:51:02.361758 23433 heartbeater.cc:347] Connected to a master server at 127.21.236.190:39493
I20260811 02:51:02.362051 23433 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:02.362685 23433 heartbeater.cc:511] Master 127.21.236.190:39493 requested a full tablet report, sending...
I20260811 02:51:02.364272 23133 ts_manager.cc:194] Registered new tserver with Master: b06f160bb556447096382c078889312a (127.21.236.129:41787)
I20260811 02:51:02.365443 23133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.129@KRBTEST.COM'} at 127.21.236.129:55299
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
Entry for principal kudu/127.21.236.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab.
I20260811 02:51:02.378173 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39493
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:02.499296 23443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:02.499552 23443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:02.499595 23443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:02.503545 23443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:02.503604 23443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:02.503661 23443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:51:02.509099 23443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.130.keytab
--principal=kudu/127.21.236.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:02.510164 23443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:02.511226 23443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:02.520927 23452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:02.520967 23450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:02.520967 23455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:02.521384 23443 server_base.cc:1034] running on GCE node
I20260811 02:51:02.521835 23443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:02.522497 23443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:02.523658 23443 hybrid_clock.cc:648] HybridClock initialized: now 1786416662523605 us; error 63 us; skew 500 ppm
Aug 11 02:51:02 dist-test-slave-n326 krb5kdc[23055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:02.526734 23443 init.cc:377] Logged in from keytab as kudu/127.21.236.130@KRBTEST.COM (short username kudu)
I20260811 02:51:02.527809 23443 webserver.cc:460] Webserver started at http://127.21.236.130:44139/ using document root <none> and password file <none>
I20260811 02:51:02.528349 23443 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:02.528406 23443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:02.528571 23443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:02.530611 23443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "9b217baa463e4451a3b26a2563480d8b"
format_stamp: "Formatted at 2026-08-11 02:51:02 on dist-test-slave-n326"
server_key: "36dd3c62bbf9f5d0f8178a972477fe7d"
server_key_iv: "65a8146507980f68c133062d685c9c40"
server_key_version: "encryptionkey@0"
I20260811 02:51:02.531214 23443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "9b217baa463e4451a3b26a2563480d8b"
format_stamp: "Formatted at 2026-08-11 02:51:02 on dist-test-slave-n326"
server_key: "36dd3c62bbf9f5d0f8178a972477fe7d"
server_key_iv: "65a8146507980f68c133062d685c9c40"
server_key_version: "encryptionkey@0"
I20260811 02:51:02.534965 23443 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 02:51:02.537595 23466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:02.538901 23443 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:51:02.539003 23443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "9b217baa463e4451a3b26a2563480d8b"
format_stamp: "Formatted at 2026-08-11 02:51:02 on dist-test-slave-n326"
server_key: "36dd3c62bbf9f5d0f8178a972477fe7d"
server_key_iv: "65a8146507980f68c133062d685c9c40"
server_key_version: "encryptionkey@0"
I20260811 02:51:02.539171 23443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:02.561904 23443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:02.565177 23443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:02.565380 23443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:02.566061 23443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:51:02.566926 23443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:02.566975 23443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:02.567014 23443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:02.567029 23443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:02.577011 23443 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:33725
I20260811 02:51:02.577033 23645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:33725 every 8 connection(s)
I20260811 02:51:02.578007 23443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:51:02.586259 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 23443
I20260811 02:51:02.586427 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:51:02.586805 22450 external_mini_cluster.cc:1428] Setting key 1cf7164891d3dffad23da0bd0e5dd457
Aug 11 02:51:02 dist-test-slave-n326 krb5kdc[23055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.130@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:51:02.592540 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:02.581395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:51799 (local address 127.21.236.190:39493)
0811 02:51:02.581563 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:02.581569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:02.581593 (+ 24us) server_negotiation.cc:407] Connection header received
0811 02:51:02.582540 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:02.582543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:02.582601 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:02.582681 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:02.583913 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.584499 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.585653 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.585881 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.586001 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:02.588915 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:02.588919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:02.588921 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:02.588946 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:02.590411 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:02.590932 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:02.590937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:02.590939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:02.591025 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:02.591426 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:02.591431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:02.591432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:02.591640 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:02.591750 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:02.592250 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 02:51:02.592376 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 02:51:02.593349 23649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:02.580504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:39493 (local address 127.21.236.130:51799)
0811 02:51:02.580960 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:02.581003 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:02.582325 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:02.582847 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:02.582853 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:02.583279 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:02.583767 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:02.583781 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.584633 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.584641 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:02.585517 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:02.585527 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.586047 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.586051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:02.586219 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:02.586850 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:02.586927 (+ 77us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:02.588748 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:02.590538 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:02.590544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:02.590558 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:02.590824 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:02.591170 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:02.591174 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:02.591176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:02.591326 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:02.591749 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:02.591756 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:02.592007 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 02:51:02.592205 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 02:51:02.592527 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":107,"threads_started":1}
I20260811 02:51:02.594530 23646 heartbeater.cc:347] Connected to a master server at 127.21.236.190:39493
I20260811 02:51:02.594821 23646 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:02.595521 23646 heartbeater.cc:511] Master 127.21.236.190:39493 requested a full tablet report, sending...
I20260811 02:51:02.596760 23133 ts_manager.cc:194] Registered new tserver with Master: 9b217baa463e4451a3b26a2563480d8b (127.21.236.130:33725)
I20260811 02:51:02.597348 23133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.130@KRBTEST.COM'} at 127.21.236.130:51799
WARNING: no policy specified for kudu/127.21.236.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
Entry for principal kudu/127.21.236.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab.
I20260811 02:51:02.618287 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39493
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:02.736635 23652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:02.736959 23652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:02.737054 23652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:02.741789 23652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:02.741909 23652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:02.742018 23652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:51:02.748420 23652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.131.keytab
--principal=kudu/127.21.236.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:39493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:02.749680 23652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:02.750823 23652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:02.761451 23660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:02.761453 23661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:02.761451 23664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:02.762109 23652 server_base.cc:1034] running on GCE node
I20260811 02:51:02.762511 23652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:02.763249 23652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:02.764497 23652 hybrid_clock.cc:648] HybridClock initialized: now 1786416662764475 us; error 38 us; skew 500 ppm
Aug 11 02:51:02 dist-test-slave-n326 krb5kdc[23055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:02.767746 23652 init.cc:377] Logged in from keytab as kudu/127.21.236.131@KRBTEST.COM (short username kudu)
I20260811 02:51:02.768673 23652 webserver.cc:460] Webserver started at http://127.21.236.131:40109/ using document root <none> and password file <none>
I20260811 02:51:02.769264 23652 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:02.769328 23652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:02.769510 23652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:02.771898 23652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "c490fe55cf38498290d397db9a60c7cf"
format_stamp: "Formatted at 2026-08-11 02:51:02 on dist-test-slave-n326"
server_key: "e8bfeb95d35bd75c34b453340324e908"
server_key_iv: "ddd02c9b76b4cc951b81d0e7e6db6338"
server_key_version: "encryptionkey@0"
I20260811 02:51:02.772476 23652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "c490fe55cf38498290d397db9a60c7cf"
format_stamp: "Formatted at 2026-08-11 02:51:02 on dist-test-slave-n326"
server_key: "e8bfeb95d35bd75c34b453340324e908"
server_key_iv: "ddd02c9b76b4cc951b81d0e7e6db6338"
server_key_version: "encryptionkey@0"
I20260811 02:51:02.776870 23652 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:51:02.779515 23673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:02.780875 23652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:02.781054 23652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "c490fe55cf38498290d397db9a60c7cf"
format_stamp: "Formatted at 2026-08-11 02:51:02 on dist-test-slave-n326"
server_key: "e8bfeb95d35bd75c34b453340324e908"
server_key_iv: "ddd02c9b76b4cc951b81d0e7e6db6338"
server_key_version: "encryptionkey@0"
I20260811 02:51:02.781179 23652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:02.816149 23652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:02.819906 23652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:02.820148 23652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:02.820820 23652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:51:02.821787 23652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:02.821842 23652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:02.821920 23652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:02.821961 23652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:02.832650 23652 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:33171
I20260811 02:51:02.832685 23844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:33171 every 8 connection(s)
I20260811 02:51:02.833662 23652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:51:02.836336 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 23652
I20260811 02:51:02.836457 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:51:02.836864 22450 external_mini_cluster.cc:1428] Setting key c295c1bff971fd761e9e791e290ec322
Aug 11 02:51:02 dist-test-slave-n326 krb5kdc[23055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.131@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:51:02.848114 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:02.835595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:41375 (local address 127.21.236.190:39493)
0811 02:51:02.835747 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:02.835752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:02.836643 (+ 891us) server_negotiation.cc:407] Connection header received
0811 02:51:02.838274 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:02.838279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:02.838349 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:02.838421 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:02.839710 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.840293 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.841424 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.841681 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.841771 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:02.844466 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:02.844469 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:02.844471 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:02.844498 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:02.845883 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:02.846459 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:02.846462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:02.846463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:02.846540 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:02.846935 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:02.846939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:02.846940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:02.847179 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:02.847280 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:02.847845 (+ 565us) server_negotiation.cc:299] Negotiation successful
0811 02:51:02.847978 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 02:51:02.848920 23849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:02.835919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:39493 (local address 127.21.236.131:41375)
0811 02:51:02.836442 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:02.836496 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:02.838077 (+ 1581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:02.838599 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:02.838606 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:02.838965 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:02.839575 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:02.839589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.840406 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.840408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:02.841295 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:02.841305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.841825 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.841828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:02.841977 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:02.842648 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:02.842666 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:02.844296 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:02.846040 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:02.846046 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:02.846059 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:02.846351 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:02.846628 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:02.846630 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:02.846632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:02.846797 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:02.847313 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:02.847319 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:02.847586 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 02:51:02.847758 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 02:51:02.848029 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":174,"threads_started":1}
I20260811 02:51:02.850220 23846 heartbeater.cc:347] Connected to a master server at 127.21.236.190:39493
I20260811 02:51:02.850482 23846 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:02.851140 23846 heartbeater.cc:511] Master 127.21.236.190:39493 requested a full tablet report, sending...
I20260811 02:51:02.852326 23133 ts_manager.cc:194] Registered new tserver with Master: c490fe55cf38498290d397db9a60c7cf (127.21.236.131:33171)
I20260811 02:51:02.852962 23133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.236.131@KRBTEST.COM'} at 127.21.236.131:41375
I20260811 02:51:02.863744 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:51:02.874624 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:02.866113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53856 (local address 127.21.236.190:39493)
0811 02:51:02.866296 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:02.866301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:02.866529 (+ 228us) server_negotiation.cc:407] Connection header received
0811 02:51:02.866708 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:02.866712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:02.866772 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:02.866869 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:02.867761 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.868361 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.869332 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.869595 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.869722 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:02.871339 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:02.871344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:02.871347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:02.871378 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:02.872833 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:02.873296 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:02.873300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:02.873302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:02.873397 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:02.873688 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:02.873692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:02.873693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:02.873936 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:02.874101 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:02.874334 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 02:51:02.874456 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:51:02.878151 23133 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:51:02.881665 23133 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:51:02.896899 23875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:02.890855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:41787 (local address 127.0.0.1:47400)
0811 02:51:02.891838 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:02.891853 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:02.891944 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:02.892809 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:02.892815 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:02.892837 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:02.893091 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:02.893100 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.894134 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.894140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:02.895418 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:02.895428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.896315 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.896318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:02.896431 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:02.896468 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 02:51:02.896624 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 02:51:02.896748 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"mutex_wait_us":101,"thread_start_us":135,"threads_started":1}
I20260811 02:51:02.896914 23880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:02.892006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47400 (local address 127.21.236.129:41787)
0811 02:51:02.892305 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:02.892311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:02.892336 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:51:02.892421 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:02.892430 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:02.892651 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:02.892808 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:02.893234 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.894009 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.895540 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.896187 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.896327 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:02.896470 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:02.896658 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 02:51:02.896756 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":123,"threads_started":1}
I20260811 02:51:02.897926 23867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:02.890617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:33725 (local address 127.0.0.1:34388)
0811 02:51:02.891044 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:02.891127 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:02.891261 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:02.892866 (+ 1605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:02.892873 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:02.892894 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:02.893159 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:02.893169 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.895026 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.895029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:02.896334 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:02.896344 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.897515 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.897520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:02.897636 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:02.897662 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 02:51:02.897734 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 02:51:02.897800 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":60,"thread_start_us":111,"threads_started":1}
I20260811 02:51:02.898264 23870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:02.891303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43770 (local address 127.21.236.131:33171)
0811 02:51:02.891622 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:02.891635 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:02.892513 (+ 878us) server_negotiation.cc:407] Connection header received
0811 02:51:02.892596 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:02.892601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:02.892770 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:02.892924 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:02.893322 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.894622 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.896338 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.897292 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.897405 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:02.897554 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:02.897982 (+ 428us) server_negotiation.cc:299] Negotiation successful
0811 02:51:02.898096 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":147,"threads_started":1}
I20260811 02:51:02.898666 23868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:02.891042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:33171 (local address 127.0.0.1:43770)
0811 02:51:02.892394 (+ 1352us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:02.892407 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:02.892505 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:02.892993 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:02.892996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:02.893010 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:02.893170 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:02.893177 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.894786 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.894790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:02.896182 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:02.896194 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.897667 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.897670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:02.897776 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:02.897801 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 02:51:02.898477 (+ 676us) client_negotiation.cc:240] Negotiation successful
0811 02:51:02.898559 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1292,"mutex_wait_us":68,"thread_start_us":120,"threads_started":1}
I20260811 02:51:02.898855 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:02.891379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34388 (local address 127.21.236.130:33725)
0811 02:51:02.892311 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:02.892324 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:02.892351 (+ 27us) server_negotiation.cc:407] Connection header received
0811 02:51:02.892438 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:02.892442 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:02.892683 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:02.893129 (+ 446us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:02.893325 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.894614 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.896509 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:02.897376 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:02.897508 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:02.897629 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:02.898521 (+ 892us) server_negotiation.cc:299] Negotiation successful
0811 02:51:02.898632 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":807,"thread_start_us":176,"threads_started":1}
I20260811 02:51:02.900607 23332 tablet_service.cc:1467] Processing CreateTablet for tablet 47288d852592435b90b520d414cef2e4 (DEFAULT_TABLE table=test-table [id=88cebf79b6fa40c38566eb8eddaa9149]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:02.901827 23332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47288d852592435b90b520d414cef2e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:02.901587 23746 tablet_service.cc:1467] Processing CreateTablet for tablet 47288d852592435b90b520d414cef2e4 (DEFAULT_TABLE table=test-table [id=88cebf79b6fa40c38566eb8eddaa9149]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:02.902606 23746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47288d852592435b90b520d414cef2e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:02.904843 23543 tablet_service.cc:1467] Processing CreateTablet for tablet 47288d852592435b90b520d414cef2e4 (DEFAULT_TABLE table=test-table [id=88cebf79b6fa40c38566eb8eddaa9149]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:02.905771 23543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47288d852592435b90b520d414cef2e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:02.909579 23885 tablet_bootstrap.cc:492] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a: Bootstrap starting.
I20260811 02:51:02.910159 23886 tablet_bootstrap.cc:492] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf: Bootstrap starting.
I20260811 02:51:02.913622 23885 tablet_bootstrap.cc:654] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:02.914191 23887 tablet_bootstrap.cc:492] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b: Bootstrap starting.
I20260811 02:51:02.914631 23885 log.cc:824] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:02.916605 23885 tablet_bootstrap.cc:492] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a: No bootstrap required, opened a new log
I20260811 02:51:02.916599 23887 tablet_bootstrap.cc:654] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:02.916617 23886 tablet_bootstrap.cc:654] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:02.916832 23885 ts_tablet_manager.cc:1397] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 02:51:02.918143 23887 log.cc:824] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:02.918143 23886 log.cc:824] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:02.920343 23887 tablet_bootstrap.cc:492] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b: No bootstrap required, opened a new log
I20260811 02:51:02.920559 23887 ts_tablet_manager.cc:1397] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 02:51:02.921000 23886 tablet_bootstrap.cc:492] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf: No bootstrap required, opened a new log
I20260811 02:51:02.921191 23886 ts_tablet_manager.cc:1397] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260811 02:51:02.920987 23885 raft_consensus.cc:348] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b217baa463e4451a3b26a2563480d8b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33725 } } peers { permanent_uuid: "b06f160bb556447096382c078889312a" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41787 } } peers { permanent_uuid: "c490fe55cf38498290d397db9a60c7cf" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33171 } }
I20260811 02:51:02.921286 23885 raft_consensus.cc:374] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:02.921353 23885 raft_consensus.cc:729] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b06f160bb556447096382c078889312a, State: Initialized, Role: FOLLOWER
I20260811 02:51:02.921835 23885 consensus_queue.cc:260] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b217baa463e4451a3b26a2563480d8b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33725 } } peers { permanent_uuid: "b06f160bb556447096382c078889312a" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41787 } } peers { permanent_uuid: "c490fe55cf38498290d397db9a60c7cf" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33171 } }
I20260811 02:51:02.922688 23885 ts_tablet_manager.cc:1428] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260811 02:51:02.923141 23433 heartbeater.cc:503] Master 127.21.236.190:39493 was elected leader, sending a full tablet report...
I20260811 02:51:02.924856 23887 raft_consensus.cc:348] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b217baa463e4451a3b26a2563480d8b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33725 } } peers { permanent_uuid: "b06f160bb556447096382c078889312a" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41787 } } peers { permanent_uuid: "c490fe55cf38498290d397db9a60c7cf" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33171 } }
I20260811 02:51:02.925161 23887 raft_consensus.cc:374] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:02.925227 23887 raft_consensus.cc:729] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b217baa463e4451a3b26a2563480d8b, State: Initialized, Role: FOLLOWER
I20260811 02:51:02.925382 23886 raft_consensus.cc:348] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b217baa463e4451a3b26a2563480d8b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33725 } } peers { permanent_uuid: "b06f160bb556447096382c078889312a" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41787 } } peers { permanent_uuid: "c490fe55cf38498290d397db9a60c7cf" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33171 } }
I20260811 02:51:02.925657 23886 raft_consensus.cc:374] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:02.925717 23886 raft_consensus.cc:729] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c490fe55cf38498290d397db9a60c7cf, State: Initialized, Role: FOLLOWER
I20260811 02:51:02.925716 23887 consensus_queue.cc:260] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b217baa463e4451a3b26a2563480d8b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33725 } } peers { permanent_uuid: "b06f160bb556447096382c078889312a" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41787 } } peers { permanent_uuid: "c490fe55cf38498290d397db9a60c7cf" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33171 } }
I20260811 02:51:02.926177 23886 consensus_queue.cc:260] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b217baa463e4451a3b26a2563480d8b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33725 } } peers { permanent_uuid: "b06f160bb556447096382c078889312a" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41787 } } peers { permanent_uuid: "c490fe55cf38498290d397db9a60c7cf" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33171 } }
I20260811 02:51:02.926837 23887 ts_tablet_manager.cc:1428] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 02:51:02.926908 23846 heartbeater.cc:503] Master 127.21.236.190:39493 was elected leader, sending a full tablet report...
I20260811 02:51:02.927196 23886 ts_tablet_manager.cc:1428] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:51:02.927227 23646 heartbeater.cc:503] Master 127.21.236.190:39493 was elected leader, sending a full tablet report...
W20260811 02:51:03.079216 23647 tablet.cc:2369] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:51:03.084820 23848 tablet.cc:2369] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:51:03.090457 23891 raft_consensus.cc:482] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:51:03.090660 23891 raft_consensus.cc:504] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b217baa463e4451a3b26a2563480d8b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33725 } } peers { permanent_uuid: "b06f160bb556447096382c078889312a" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41787 } } peers { permanent_uuid: "c490fe55cf38498290d397db9a60c7cf" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33171 } }
I20260811 02:51:03.091864 23891 leader_election.cc:288] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b217baa463e4451a3b26a2563480d8b (127.21.236.130:33725), c490fe55cf38498290d397db9a60c7cf (127.21.236.131:33171)
W20260811 02:51:03.095216 23438 tablet.cc:2369] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:51:03.096783 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:03.092246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:35945 (local address 127.21.236.130:33725)
0811 02:51:03.092404 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:03.092410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:03.092678 (+ 268us) server_negotiation.cc:407] Connection header received
0811 02:51:03.092859 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:03.092867 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:03.092914 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:03.093004 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:03.093306 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.093815 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.095333 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.096098 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.096212 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:03.096253 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:03.096567 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 02:51:03.096648 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:51:03.096783 23870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:03.092453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:48057 (local address 127.21.236.131:33171)
0811 02:51:03.092581 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:03.092589 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:03.092756 (+ 167us) server_negotiation.cc:407] Connection header received
0811 02:51:03.092949 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:03.092954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:03.093011 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:03.093107 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:03.093457 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.093987 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.095201 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.095929 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.096192 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:03.096250 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:03.096567 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 02:51:03.096648 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:51:03.096920 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:03.092096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:33725 (local address 127.21.236.129:35945)
0811 02:51:03.092560 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:03.092576 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:03.092742 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:03.093024 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:03.093026 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:03.093044 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:03.093189 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:03.093194 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.093939 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.093946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:03.095119 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:03.095131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.096223 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.096227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:03.096388 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:03.096420 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 02:51:03.096529 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 02:51:03.096648 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":150,"thread_start_us":171,"threads_started":1}
I20260811 02:51:03.097102 23894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:03.092243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:33171 (local address 127.21.236.129:48057)
0811 02:51:03.092647 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:03.092661 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:03.092814 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:03.093123 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:03.093126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:03.093146 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:03.093279 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:03.093285 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.094119 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.094123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:03.095030 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:03.095040 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.096220 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.096224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:03.096426 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:03.096453 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 02:51:03.096531 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 02:51:03.096648 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":28,"thread_start_us":114,"threads_started":1}
I20260811 02:51:03.097693 23577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47288d852592435b90b520d414cef2e4" candidate_uuid: "b06f160bb556447096382c078889312a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b217baa463e4451a3b26a2563480d8b" is_pre_election: true
I20260811 02:51:03.097846 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47288d852592435b90b520d414cef2e4" candidate_uuid: "b06f160bb556447096382c078889312a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c490fe55cf38498290d397db9a60c7cf" is_pre_election: true
I20260811 02:51:03.098016 23577 raft_consensus.cc:2454] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b06f160bb556447096382c078889312a in term 0.
I20260811 02:51:03.098194 23780 raft_consensus.cc:2454] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b06f160bb556447096382c078889312a in term 0.
I20260811 02:51:03.098619 23258 leader_election.cc:302] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b217baa463e4451a3b26a2563480d8b, b06f160bb556447096382c078889312a; no voters:
I20260811 02:51:03.098940 23891 raft_consensus.cc:2781] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:51:03.099046 23891 raft_consensus.cc:482] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:51:03.099148 23891 raft_consensus.cc:3037] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:03.100257 23891 raft_consensus.cc:504] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b217baa463e4451a3b26a2563480d8b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33725 } } peers { permanent_uuid: "b06f160bb556447096382c078889312a" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41787 } } peers { permanent_uuid: "c490fe55cf38498290d397db9a60c7cf" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33171 } }
I20260811 02:51:03.100631 23891 leader_election.cc:288] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [CANDIDATE]: Term 1 election: Requested vote from peers 9b217baa463e4451a3b26a2563480d8b (127.21.236.130:33725), c490fe55cf38498290d397db9a60c7cf (127.21.236.131:33171)
I20260811 02:51:03.101122 23780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47288d852592435b90b520d414cef2e4" candidate_uuid: "b06f160bb556447096382c078889312a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c490fe55cf38498290d397db9a60c7cf"
I20260811 02:51:03.101123 23577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47288d852592435b90b520d414cef2e4" candidate_uuid: "b06f160bb556447096382c078889312a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b217baa463e4451a3b26a2563480d8b"
I20260811 02:51:03.101261 23780 raft_consensus.cc:3037] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:03.101262 23577 raft_consensus.cc:3037] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:03.102622 23577 raft_consensus.cc:2454] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b06f160bb556447096382c078889312a in term 1.
I20260811 02:51:03.102622 23780 raft_consensus.cc:2454] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b06f160bb556447096382c078889312a in term 1.
I20260811 02:51:03.102998 23258 leader_election.cc:302] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b217baa463e4451a3b26a2563480d8b, b06f160bb556447096382c078889312a; no voters:
I20260811 02:51:03.103271 23891 raft_consensus.cc:2781] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:03.103518 23891 raft_consensus.cc:686] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [term 1 LEADER]: Becoming Leader. State: Replica: b06f160bb556447096382c078889312a, State: Running, Role: LEADER
I20260811 02:51:03.103889 23891 consensus_queue.cc:237] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b217baa463e4451a3b26a2563480d8b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33725 } } peers { permanent_uuid: "b06f160bb556447096382c078889312a" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41787 } } peers { permanent_uuid: "c490fe55cf38498290d397db9a60c7cf" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33171 } }
I20260811 02:51:03.107240 23133 catalog_manager.cc:5495] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a reported cstate change: term changed from 0 to 1, leader changed from <none> to b06f160bb556447096382c078889312a (127.21.236.129). New cstate: current_term: 1 leader_uuid: "b06f160bb556447096382c078889312a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b217baa463e4451a3b26a2563480d8b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b06f160bb556447096382c078889312a" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 41787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c490fe55cf38498290d397db9a60c7cf" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33171 } health_report { overall_health: UNKNOWN } } }
I20260811 02:51:03.133551 23880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:03.129469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47406 (local address 127.21.236.129:41787)
0811 02:51:03.129632 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:03.129637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:03.129654 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:51:03.129810 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:03.129813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:03.129882 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:03.130021 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:03.130302 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.130860 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.131952 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.132190 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.132312 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:03.132669 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:03.133113 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:03.133261 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:03.133328 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 02:51:03.133389 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 02:51:03.139703 23780 raft_consensus.cc:1261] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf [term 1 FOLLOWER]: Refusing update from remote peer b06f160bb556447096382c078889312a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:03.139703 23577 raft_consensus.cc:1261] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b [term 1 FOLLOWER]: Refusing update from remote peer b06f160bb556447096382c078889312a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:03.140455 23896 consensus_queue.cc:1035] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b217baa463e4451a3b26a2563480d8b" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:03.140712 23891 consensus_queue.cc:1035] T 47288d852592435b90b520d414cef2e4 P b06f160bb556447096382c078889312a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c490fe55cf38498290d397db9a60c7cf" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 33171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:03.148685 23904 mvcc.cc:205] Tried to move back new op lower bound from 7317162652212207616 to 7317162652076175360. Current Snapshot: MvccSnapshot[applied={T|T < 7317162652212207616}]
I20260811 02:51:03.150262 23903 mvcc.cc:205] Tried to move back new op lower bound from 7317162652212207616 to 7317162652076175360. Current Snapshot: MvccSnapshot[applied={T|T < 7317162652212207616}]
I20260811 02:51:03.157258 23133 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:51:03.157419 23133 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:51:03.160014 23133 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5169c3619c24a9696c4d3ed0a7e9a79: Sending DeleteTablet for 3 replicas of tablet 47288d852592435b90b520d414cef2e4
I20260811 02:51:03.160928 23332 tablet_service.cc:1514] Processing DeleteTablet for tablet 47288d852592435b90b520d414cef2e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:03 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:47400
I20260811 02:51:03.160925 23543 tablet_service.cc:1514] Processing DeleteTablet for tablet 47288d852592435b90b520d414cef2e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:03 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:34388
I20260811 02:51:03.161057 23746 tablet_service.cc:1514] Processing DeleteTablet for tablet 47288d852592435b90b520d414cef2e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:03 UTC) from {username='kudu', principal='kudu/127.21.236.190@KRBTEST.COM'} at 127.0.0.1:43770
I20260811 02:51:03.161510 23923 tablet_replica.cc:330] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b: stopping tablet replica
I20260811 02:51:03.161707 23926 tablet_replica.cc:330] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf: stopping tablet replica
I20260811 02:51:03.161883 23923 raft_consensus.cc:2229] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:51:03.161939 23926 raft_consensus.cc:2229] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:51:03.162204 23923 raft_consensus.cc:2258] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:51:03.162204 23926 raft_consensus.cc:2258] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:51:03.162292 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 23230
I20260811 02:51:03.164032 23926 ts_tablet_manager.cc:1905] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:03.168874 23926 ts_tablet_manager.cc:1918] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:51:03.168972 23926 log.cc:1196] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/47288d852592435b90b520d414cef2e4
I20260811 02:51:03.169322 23926 ts_tablet_manager.cc:1939] T 47288d852592435b90b520d414cef2e4 P c490fe55cf38498290d397db9a60c7cf: Deleting consensus metadata
I20260811 02:51:03.170586 23105 catalog_manager.cc:4841] TS c490fe55cf38498290d397db9a60c7cf (127.21.236.131:33171): tablet 47288d852592435b90b520d414cef2e4 (table test-table [id=88cebf79b6fa40c38566eb8eddaa9149]) successfully deleted
I20260811 02:51:03.172001 23923 ts_tablet_manager.cc:1905] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:03.174156 23923 ts_tablet_manager.cc:1918] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:51:03.174283 23923 log.cc:1196] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/47288d852592435b90b520d414cef2e4
I20260811 02:51:03.174605 23923 ts_tablet_manager.cc:1939] T 47288d852592435b90b520d414cef2e4 P 9b217baa463e4451a3b26a2563480d8b: Deleting consensus metadata
I20260811 02:51:03.175872 23106 catalog_manager.cc:4841] TS 9b217baa463e4451a3b26a2563480d8b (127.21.236.130:33725): tablet 47288d852592435b90b520d414cef2e4 (table test-table [id=88cebf79b6fa40c38566eb8eddaa9149]) successfully deleted
W20260811 02:51:03.177088 23111 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:41787 (error 108)
I20260811 02:51:03.177320 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 23443
I20260811 02:51:03.185170 23868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:03.184754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:41787 (local address 127.0.0.1:47418)
0811 02:51:03.184977 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:03.185060 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:41787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260811 02:51:03.185405 23111 catalog_manager.cc:4584] TS b06f160bb556447096382c078889312a (127.21.236.129:41787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 47288d852592435b90b520d414cef2e4: Network error: Client connection negotiation failed: client connection to 127.21.236.129:41787: connect: Connection refused (error 111)
I20260811 02:51:03.194649 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 23652
I20260811 02:51:03.206140 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 23076
2026-08-11T02:51:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2837 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T02:51:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:03Z Disabled control of system clock
I20260811 02:51:03.247435 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:42705
--webserver_interface=127.21.236.190
--webserver_port=0
--builtin_ntp_servers=127.21.236.148:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:42705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 02:51:03.365116 23939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:03.365391 23939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:03.365428 23939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:03.369514 23939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:03.369575 23939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:03.369590 23939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:03.369607 23939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:03.369622 23939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:03.374512 23939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:42705
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:42705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:03.375582 23939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:03.376915 23939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:03.385488 23947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:03.385489 23951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:03.385708 23939 server_base.cc:1034] running on GCE node
W20260811 02:51:03.385489 23949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:03.386405 23939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:03.387674 23939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:03.388890 23939 hybrid_clock.cc:648] HybridClock initialized: now 1786416663388853 us; error 47 us; skew 500 ppm
I20260811 02:51:03.395504 23939 webserver.cc:460] Webserver started at http://127.21.236.190:42141/ using document root <none> and password file <none>
I20260811 02:51:03.396190 23939 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:03.396298 23939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:03.396569 23939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:03.398993 23939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "afe12713a1df43f48bb428bb2bbc03ae"
format_stamp: "Formatted at 2026-08-11 02:51:03 on dist-test-slave-n326"
server_key: "413556ed224a379481f256049999d590"
server_key_iv: "7b9c242de056c0a5c0a0cfa865d08bc6"
server_key_version: "encryptionkey@0"
I20260811 02:51:03.399685 23939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "afe12713a1df43f48bb428bb2bbc03ae"
format_stamp: "Formatted at 2026-08-11 02:51:03 on dist-test-slave-n326"
server_key: "413556ed224a379481f256049999d590"
server_key_iv: "7b9c242de056c0a5c0a0cfa865d08bc6"
server_key_version: "encryptionkey@0"
I20260811 02:51:03.403724 23939 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:51:03.406658 23957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:03.408119 23939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:03.408304 23939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "afe12713a1df43f48bb428bb2bbc03ae"
format_stamp: "Formatted at 2026-08-11 02:51:03 on dist-test-slave-n326"
server_key: "413556ed224a379481f256049999d590"
server_key_iv: "7b9c242de056c0a5c0a0cfa865d08bc6"
server_key_version: "encryptionkey@0"
I20260811 02:51:03.408445 23939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:03.441219 23939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:03.442025 23939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:03.442250 23939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:03.449946 23939 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:42705
I20260811 02:51:03.449991 24043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:42705 every 8 connection(s)
I20260811 02:51:03.450958 23939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:03.454607 24045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:03.454813 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 23939
I20260811 02:51:03.454952 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:03.455317 22450 external_mini_cluster.cc:1428] Setting key 6b1f7cc708601dbeabd87c2eb3b3ffba
I20260811 02:51:03.463011 24045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae: Bootstrap starting.
I20260811 02:51:03.466109 24045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:03.467140 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:03.456682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36912 (local address 127.21.236.190:42705)
0811 02:51:03.457400 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:03.457422 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:03.457468 (+ 46us) server_negotiation.cc:407] Connection header received
0811 02:51:03.458697 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:03.458724 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:03.459317 (+ 593us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:03.459723 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:03.460217 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.461928 (+ 1711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.462894 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.463646 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.463823 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:03.464794 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:03.464801 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:03.464818 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:03.465347 (+ 529us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:03.465427 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:03.465455 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:03.465563 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:03.465668 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 02:51:03.465996 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":131,"threads_started":1}
I20260811 02:51:03.467140 24045 log.cc:824] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:03.469460 24045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae: No bootstrap required, opened a new log
I20260811 02:51:03.473168 24045 raft_consensus.cc:348] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe12713a1df43f48bb428bb2bbc03ae" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42705 } }
I20260811 02:51:03.473400 24045 raft_consensus.cc:374] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:03.473448 24045 raft_consensus.cc:729] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afe12713a1df43f48bb428bb2bbc03ae, State: Initialized, Role: FOLLOWER
I20260811 02:51:03.474056 24045 consensus_queue.cc:260] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe12713a1df43f48bb428bb2bbc03ae" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42705 } }
I20260811 02:51:03.474217 24045 raft_consensus.cc:388] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:03.474272 24045 raft_consensus.cc:482] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:03.474345 24045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:03.475587 24045 raft_consensus.cc:504] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe12713a1df43f48bb428bb2bbc03ae" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42705 } }
I20260811 02:51:03.475904 24045 leader_election.cc:302] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afe12713a1df43f48bb428bb2bbc03ae; no voters:
I20260811 02:51:03.476178 24045 leader_election.cc:288] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:03.476329 24053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:03.476617 24053 raft_consensus.cc:686] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [term 1 LEADER]: Becoming Leader. State: Replica: afe12713a1df43f48bb428bb2bbc03ae, State: Running, Role: LEADER
I20260811 02:51:03.476915 24053 consensus_queue.cc:237] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe12713a1df43f48bb428bb2bbc03ae" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42705 } }
I20260811 02:51:03.477451 24045 sys_catalog.cc:564] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:03.478669 24055 sys_catalog.cc:455] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader afe12713a1df43f48bb428bb2bbc03ae. Latest consensus state: current_term: 1 leader_uuid: "afe12713a1df43f48bb428bb2bbc03ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe12713a1df43f48bb428bb2bbc03ae" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42705 } } }
I20260811 02:51:03.478808 24055 sys_catalog.cc:458] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:03.479137 24054 sys_catalog.cc:455] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afe12713a1df43f48bb428bb2bbc03ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe12713a1df43f48bb428bb2bbc03ae" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 42705 } } }
I20260811 02:51:03.479241 24054 sys_catalog.cc:458] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:03.479355 24069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:03.482091 24069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:03.488073 24069 catalog_manager.cc:1347] Generated new cluster ID: 07b4262a17e5407eaa66e15e492eab2a
I20260811 02:51:03.488147 24069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:03.508960 24069 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:03.510056 24069 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:03.516537 24069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae: Generated new TSK 0
I20260811 02:51:03.517360 24069 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:03.521157 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.129:0
--local_ip_for_outbound_sockets=127.21.236.129
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:42705
--builtin_ntp_servers=127.21.236.148:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 02:51:03.636291 24089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:03.636557 24089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:03.636600 24089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:03.640844 24089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:03.640911 24089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:03.641000 24089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.129
I20260811 02:51:03.645838 24089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.236.129
--webserver_port=0
--tserver_master_addrs=127.21.236.190:42705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.129
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:03.646962 24089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:03.648591 24089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:03.659178 24096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:03.659179 24097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:03.659180 24101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:03.659451 24089 server_base.cc:1034] running on GCE node
I20260811 02:51:03.659833 24089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:03.660406 24089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:03.661599 24089 hybrid_clock.cc:648] HybridClock initialized: now 1786416663661587 us; error 36 us; skew 500 ppm
I20260811 02:51:03.663584 24089 webserver.cc:460] Webserver started at http://127.21.236.129:35685/ using document root <none> and password file <none>
I20260811 02:51:03.664186 24089 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:03.664284 24089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:03.664522 24089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:03.666568 24089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/data/instance:
uuid: "b9410ac24fef470598e2c58f2588f7bf"
format_stamp: "Formatted at 2026-08-11 02:51:03 on dist-test-slave-n326"
server_key: "a1f5fefa2e7516a4e6d621804878f69b"
server_key_iv: "50134e7f687fe288386623753307a9d5"
server_key_version: "encryptionkey@0"
I20260811 02:51:03.667260 24089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance:
uuid: "b9410ac24fef470598e2c58f2588f7bf"
format_stamp: "Formatted at 2026-08-11 02:51:03 on dist-test-slave-n326"
server_key: "a1f5fefa2e7516a4e6d621804878f69b"
server_key_iv: "50134e7f687fe288386623753307a9d5"
server_key_version: "encryptionkey@0"
I20260811 02:51:03.670868 24089 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:51:03.673300 24111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:03.674476 24089 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:51:03.674618 24089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/wal
uuid: "b9410ac24fef470598e2c58f2588f7bf"
format_stamp: "Formatted at 2026-08-11 02:51:03 on dist-test-slave-n326"
server_key: "a1f5fefa2e7516a4e6d621804878f69b"
server_key_iv: "50134e7f687fe288386623753307a9d5"
server_key_version: "encryptionkey@0"
I20260811 02:51:03.674732 24089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:03.703470 24089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:03.704241 24089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:03.704457 24089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:03.705087 24089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:51:03.706081 24089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:03.706157 24089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:03.706230 24089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:03.706272 24089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:03.717321 24089 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.129:42011
I20260811 02:51:03.717351 24317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.129:42011 every 8 connection(s)
I20260811 02:51:03.718292 24089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/data/info.pb
I20260811 02:51:03.719178 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 24089
I20260811 02:51:03.719323 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-0/wal/instance
I20260811 02:51:03.719627 22450 external_mini_cluster.cc:1428] Setting key 8bdfd4d0045f3c8eccfc0baa6252dcb1
I20260811 02:51:03.721740 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.130:0
--local_ip_for_outbound_sockets=127.21.236.130
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:42705
--builtin_ntp_servers=127.21.236.148:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 02:51:03.729965 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:03.720526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.129:50801 (local address 127.21.236.190:42705)
0811 02:51:03.720667 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:03.720672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:03.721876 (+ 1204us) server_negotiation.cc:407] Connection header received
0811 02:51:03.723520 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:03.723523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:03.723580 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:03.723656 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:03.724864 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.725383 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.727369 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.727626 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.727731 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:03.729421 (+ 1690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:03.729424 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:03.729426 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:03.729440 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:03.729461 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:03.729470 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:03.729534 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:03.729777 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 02:51:03.729843 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 02:51:03.731400 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:03.720912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:42705 (local address 127.21.236.129:50801)
0811 02:51:03.721728 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:03.721780 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:03.723243 (+ 1463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:03.723862 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:03.723871 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:03.723986 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:03.724716 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:03.724737 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.725525 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.725532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:03.727200 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:03.727214 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.727831 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.727838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:03.728089 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:03.729046 (+ 957us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:51:03.729051 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:03.729120 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:51:03.729221 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:03.729574 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:03.729589 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:03.729597 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 02:51:03.729764 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 02:51:03.730133 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":169,"threads_started":1}
I20260811 02:51:03.732851 24318 heartbeater.cc:347] Connected to a master server at 127.21.236.190:42705
I20260811 02:51:03.733121 24318 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:03.733688 24318 heartbeater.cc:511] Master 127.21.236.190:42705 requested a full tablet report, sending...
I20260811 02:51:03.735312 23991 ts_manager.cc:194] Registered new tserver with Master: b9410ac24fef470598e2c58f2588f7bf (127.21.236.129:42011)
I20260811 02:51:03.736538 23991 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.236.129:50801
W20260811 02:51:03.840133 24326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:03.840481 24326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:03.840565 24326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:03.844805 24326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:03.844893 24326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:03.844969 24326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.130
I20260811 02:51:03.849810 24326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.236.130
--webserver_port=0
--tserver_master_addrs=127.21.236.190:42705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.130
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:03.850891 24326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:03.852430 24326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:03.862350 24336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:03.862435 24326 server_base.cc:1034] running on GCE node
W20260811 02:51:03.862361 24333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:03.862350 24338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:03.863044 24326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:03.863799 24326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:03.864987 24326 hybrid_clock.cc:648] HybridClock initialized: now 1786416663864973 us; error 26 us; skew 500 ppm
I20260811 02:51:03.866917 24326 webserver.cc:460] Webserver started at http://127.21.236.130:46029/ using document root <none> and password file <none>
I20260811 02:51:03.867643 24326 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:03.867743 24326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:03.867983 24326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:03.870108 24326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/data/instance:
uuid: "252b32abb02f4e54a2abdabb7a9bee5f"
format_stamp: "Formatted at 2026-08-11 02:51:03 on dist-test-slave-n326"
server_key: "c63dd8e9375b90770fd15c0217249341"
server_key_iv: "b9349b9c0519b9eef282595e56cf0df4"
server_key_version: "encryptionkey@0"
I20260811 02:51:03.870702 24326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance:
uuid: "252b32abb02f4e54a2abdabb7a9bee5f"
format_stamp: "Formatted at 2026-08-11 02:51:03 on dist-test-slave-n326"
server_key: "c63dd8e9375b90770fd15c0217249341"
server_key_iv: "b9349b9c0519b9eef282595e56cf0df4"
server_key_version: "encryptionkey@0"
I20260811 02:51:03.874465 24326 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.001s
I20260811 02:51:03.877034 24347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:03.878525 24326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:51:03.878624 24326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/wal
uuid: "252b32abb02f4e54a2abdabb7a9bee5f"
format_stamp: "Formatted at 2026-08-11 02:51:03 on dist-test-slave-n326"
server_key: "c63dd8e9375b90770fd15c0217249341"
server_key_iv: "b9349b9c0519b9eef282595e56cf0df4"
server_key_version: "encryptionkey@0"
I20260811 02:51:03.878767 24326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:03.900614 24326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:03.901330 24326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:03.901566 24326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:03.902204 24326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:51:03.903247 24326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:03.903301 24326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:03.903343 24326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:03.903379 24326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:03.912679 24326 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.130:33109
I20260811 02:51:03.912707 24535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.130:33109 every 8 connection(s)
I20260811 02:51:03.913623 24326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/data/info.pb
I20260811 02:51:03.920076 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 24326
I20260811 02:51:03.920248 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/wal/instance
I20260811 02:51:03.920533 22450 external_mini_cluster.cc:1428] Setting key ec17f2c31d71ba5d25fb76283d0eb96b
I20260811 02:51:03.922528 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.236.131:0
--local_ip_for_outbound_sockets=127.21.236.131
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:42705
--builtin_ntp_servers=127.21.236.148:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 02:51:03.923826 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:03.915494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.130:52993 (local address 127.21.236.190:42705)
0811 02:51:03.915690 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:03.915695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:03.916764 (+ 1069us) server_negotiation.cc:407] Connection header received
0811 02:51:03.918142 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:03.918146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:03.918195 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:03.918283 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:03.919357 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.919955 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.921511 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.921862 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.921990 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:03.923259 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:03.923264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:03.923267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:03.923281 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:03.923312 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:03.923322 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:03.923417 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:03.923596 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 02:51:03.923679 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 02:51:03.924711 24540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:03.915908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:42705 (local address 127.21.236.130:52993)
0811 02:51:03.916607 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:03.916649 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:03.917940 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:03.918501 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:03.918508 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:03.918596 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:03.919196 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:03.919214 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.920108 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.920111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:03.921373 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:03.921384 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:03.922032 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:03.922037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:03.922226 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:03.922921 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:51:03.922923 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:03.922966 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:51:03.923033 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:03.923426 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:03.923438 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:03.923445 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 02:51:03.923595 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 02:51:03.923891 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":128,"threads_started":1}
I20260811 02:51:03.925952 24538 heartbeater.cc:347] Connected to a master server at 127.21.236.190:42705
I20260811 02:51:03.926214 24538 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:03.926822 24538 heartbeater.cc:511] Master 127.21.236.190:42705 requested a full tablet report, sending...
I20260811 02:51:03.928145 23991 ts_manager.cc:194] Registered new tserver with Master: 252b32abb02f4e54a2abdabb7a9bee5f (127.21.236.130:33109)
I20260811 02:51:03.929033 23991 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.236.130:52993
W20260811 02:51:04.042989 24544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:04.043286 24544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:04.043327 24544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:04.047276 24544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:04.047325 24544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:04.047379 24544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.236.131
I20260811 02:51:04.052289 24544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.236.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.236.131
--webserver_port=0
--tserver_master_addrs=127.21.236.190:42705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.236.131
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:04.053329 24544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:04.054661 24544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:04.064498 24559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:04.064827 24555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:04.064769 24544 server_base.cc:1034] running on GCE node
W20260811 02:51:04.064882 24557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:04.065285 24544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:04.065944 24544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:04.067144 24544 hybrid_clock.cc:648] HybridClock initialized: now 1786416664067131 us; error 26 us; skew 500 ppm
I20260811 02:51:04.068982 24544 webserver.cc:460] Webserver started at http://127.21.236.131:38067/ using document root <none> and password file <none>
I20260811 02:51:04.069579 24544 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:04.069677 24544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:04.069919 24544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:04.072000 24544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/data/instance:
uuid: "eac88b0486d2430580f98914d6781d05"
format_stamp: "Formatted at 2026-08-11 02:51:04 on dist-test-slave-n326"
server_key: "ae8db47a97674d62a28142ceb31d851d"
server_key_iv: "0181b8ae6a8b7d592c9b769086a07244"
server_key_version: "encryptionkey@0"
I20260811 02:51:04.072569 24544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance:
uuid: "eac88b0486d2430580f98914d6781d05"
format_stamp: "Formatted at 2026-08-11 02:51:04 on dist-test-slave-n326"
server_key: "ae8db47a97674d62a28142ceb31d851d"
server_key_iv: "0181b8ae6a8b7d592c9b769086a07244"
server_key_version: "encryptionkey@0"
I20260811 02:51:04.076262 24544 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 02:51:04.078850 24568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:04.080363 24544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:51:04.080507 24544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/wal
uuid: "eac88b0486d2430580f98914d6781d05"
format_stamp: "Formatted at 2026-08-11 02:51:04 on dist-test-slave-n326"
server_key: "ae8db47a97674d62a28142ceb31d851d"
server_key_iv: "0181b8ae6a8b7d592c9b769086a07244"
server_key_version: "encryptionkey@0"
I20260811 02:51:04.080643 24544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:04.096359 24544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:04.097038 24544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:04.097242 24544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:04.097847 24544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 02:51:04.098799 24544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 02:51:04.098873 24544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:04.098939 24544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 02:51:04.098986 24544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:04.107884 24544 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.131:42039
I20260811 02:51:04.107919 24758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.131:42039 every 8 connection(s)
I20260811 02:51:04.108758 24544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/data/info.pb
I20260811 02:51:04.110586 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 24544
I20260811 02:51:04.110711 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/wal/instance
I20260811 02:51:04.111023 22450 external_mini_cluster.cc:1428] Setting key 84a79e50bd4d674888ab68e49937af37
I20260811 02:51:04.118628 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.110695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:53977 (local address 127.21.236.190:42705)
0811 02:51:04.110823 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:04.110827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:04.112201 (+ 1374us) server_negotiation.cc:407] Connection header received
0811 02:51:04.113932 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:04.113935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:04.113993 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:04.114094 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:04.114952 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.115585 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.116729 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.116964 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.117082 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:04.118119 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:04.118123 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:04.118126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:04.118138 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:04.118165 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:04.118174 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:04.118254 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:04.118418 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 02:51:04.118487 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 02:51:04.119486 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.111241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:42705 (local address 127.21.236.131:53977)
0811 02:51:04.112063 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:04.112120 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:04.113730 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:04.114269 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:04.114276 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:04.114357 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:04.114816 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:04.114827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.115723 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.115727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:04.116598 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:04.116608 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.117122 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.117126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:04.117295 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:04.117853 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 02:51:04.117855 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:04.117893 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 02:51:04.117968 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:04.118256 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:04.118267 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:04.118275 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 02:51:04.118404 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 02:51:04.118672 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":125,"threads_started":1}
I20260811 02:51:04.120563 24759 heartbeater.cc:347] Connected to a master server at 127.21.236.190:42705
I20260811 02:51:04.120774 24759 heartbeater.cc:464] Registering TS with master...
I20260811 02:51:04.121256 24759 heartbeater.cc:511] Master 127.21.236.190:42705 requested a full tablet report, sending...
I20260811 02:51:04.122241 23991 ts_manager.cc:194] Registered new tserver with Master: eac88b0486d2430580f98914d6781d05 (127.21.236.131:42039)
I20260811 02:51:04.122798 23991 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.236.131:53977
I20260811 02:51:04.125265 22450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 02:51:04.131235 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.127245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36924 (local address 127.21.236.190:42705)
0811 02:51:04.127358 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:04.127368 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:04.127666 (+ 298us) server_negotiation.cc:407] Connection header received
0811 02:51:04.127709 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:04.127711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:04.127752 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:04.127823 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:04.128157 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.128650 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.129595 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.129817 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.129904 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:04.130809 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:04.130812 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:04.130814 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:04.130826 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:04.130844 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:04.130852 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:04.130913 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:04.131016 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 02:51:04.131118 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 02:51:04.134261 23991 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 02:51:04.137701 23991 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 02:51:04.153529 24779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.146742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48564 (local address 127.21.236.131:42039)
0811 02:51:04.148384 (+ 1642us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:04.148395 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:04.148417 (+ 22us) server_negotiation.cc:407] Connection header received
0811 02:51:04.148495 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:04.148502 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:04.148673 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:04.149095 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:04.149232 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.150300 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.151757 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.152579 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.152712 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:04.152892 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:04.153248 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 02:51:04.153353 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1541,"thread_start_us":216,"threads_started":1}
I20260811 02:51:04.153542 24781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.147306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:33109 (local address 127.0.0.1:37516)
0811 02:51:04.148022 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:04.148035 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:04.148125 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:04.148589 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:04.148592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:04.148610 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:04.148781 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:04.148787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.150180 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.150184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:04.151732 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:04.151744 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.152746 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.152750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:04.152988 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:04.153077 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 02:51:04.153199 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 02:51:04.153360 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":217,"thread_start_us":117,"threads_started":1}
I20260811 02:51:04.153542 24778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.146563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.131:42039 (local address 127.0.0.1:48564)
0811 02:51:04.147105 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:04.147156 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:04.147289 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:04.148822 (+ 1533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:04.148825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:04.148843 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:04.148949 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:04.148955 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.150420 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.150425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:04.151629 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:04.151639 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.152746 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.152750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:04.153006 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:04.153077 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 02:51:04.153199 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 02:51:04.153364 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":186,"thread_start_us":86,"threads_started":1}
I20260811 02:51:04.154191 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.147669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37516 (local address 127.21.236.130:33109)
0811 02:51:04.148112 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:04.148123 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:04.148149 (+ 26us) server_negotiation.cc:407] Connection header received
0811 02:51:04.148229 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:04.148236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:04.148434 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:04.148690 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:04.148882 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.150059 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.151849 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.152579 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.152712 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:04.152891 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:04.153769 (+ 878us) server_negotiation.cc:299] Negotiation successful
0811 02:51:04.154035 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":121,"threads_started":1}
I20260811 02:51:04.154220 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.147043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42011 (local address 127.0.0.1:36578)
0811 02:51:04.147706 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:04.147720 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:04.147816 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:04.148463 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:04.148470 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:04.148499 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:04.148710 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:04.148719 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.149969 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.149974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:04.151732 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:04.151744 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.153850 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.153854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:04.153938 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:04.153962 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 02:51:04.154017 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 02:51:04.154086 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":130,"thread_start_us":103,"threads_started":1}
I20260811 02:51:04.155632 24783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.147420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36578 (local address 127.21.236.129:42011)
0811 02:51:04.147741 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:04.147754 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:04.147786 (+ 32us) server_negotiation.cc:407] Connection header received
0811 02:51:04.148003 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:04.148011 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:04.148242 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:04.148899 (+ 657us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:04.149059 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.149838 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.152968 (+ 3130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.153701 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.155270 (+ 1569us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:04.155406 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:04.155444 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 02:51:04.155510 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":118,"threads_started":1}
I20260811 02:51:04.157394 24427 tablet_service.cc:1467] Processing CreateTablet for tablet 73c36745ec544c8aba0c9becb65f2515 (DEFAULT_TABLE table=test-table [id=8420a7a0dab14d5cb3161704723df003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:04.157397 24653 tablet_service.cc:1467] Processing CreateTablet for tablet 73c36745ec544c8aba0c9becb65f2515 (DEFAULT_TABLE table=test-table [id=8420a7a0dab14d5cb3161704723df003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:04.157712 24198 tablet_service.cc:1467] Processing CreateTablet for tablet 73c36745ec544c8aba0c9becb65f2515 (DEFAULT_TABLE table=test-table [id=8420a7a0dab14d5cb3161704723df003]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 02:51:04.158562 24427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73c36745ec544c8aba0c9becb65f2515. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:04.158562 24653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73c36745ec544c8aba0c9becb65f2515. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:04.158798 24198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73c36745ec544c8aba0c9becb65f2515. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:04.165979 24785 tablet_bootstrap.cc:492] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f: Bootstrap starting.
I20260811 02:51:04.166396 24787 tablet_bootstrap.cc:492] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05: Bootstrap starting.
I20260811 02:51:04.167356 24789 tablet_bootstrap.cc:492] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf: Bootstrap starting.
I20260811 02:51:04.169019 24787 tablet_bootstrap.cc:654] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:04.169353 24785 tablet_bootstrap.cc:654] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:04.169601 24789 tablet_bootstrap.cc:654] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:04.169960 24787 log.cc:824] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:04.170207 24785 log.cc:824] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:04.170408 24789 log.cc:824] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:04.172120 24787 tablet_bootstrap.cc:492] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05: No bootstrap required, opened a new log
I20260811 02:51:04.172273 24785 tablet_bootstrap.cc:492] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f: No bootstrap required, opened a new log
I20260811 02:51:04.172333 24787 ts_tablet_manager.cc:1397] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 02:51:04.172515 24785 ts_tablet_manager.cc:1397] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 02:51:04.176312 24785 raft_consensus.cc:348] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac88b0486d2430580f98914d6781d05" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 42039 } } peers { permanent_uuid: "252b32abb02f4e54a2abdabb7a9bee5f" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33109 } } peers { permanent_uuid: "b9410ac24fef470598e2c58f2588f7bf" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42011 } }
I20260811 02:51:04.176553 24789 tablet_bootstrap.cc:492] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf: No bootstrap required, opened a new log
I20260811 02:51:04.176585 24785 raft_consensus.cc:374] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:04.176432 24787 raft_consensus.cc:348] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac88b0486d2430580f98914d6781d05" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 42039 } } peers { permanent_uuid: "252b32abb02f4e54a2abdabb7a9bee5f" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33109 } } peers { permanent_uuid: "b9410ac24fef470598e2c58f2588f7bf" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42011 } }
I20260811 02:51:04.176645 24785 raft_consensus.cc:729] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 252b32abb02f4e54a2abdabb7a9bee5f, State: Initialized, Role: FOLLOWER
I20260811 02:51:04.176681 24787 raft_consensus.cc:374] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:04.176741 24789 ts_tablet_manager.cc:1397] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260811 02:51:04.176728 24787 raft_consensus.cc:729] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac88b0486d2430580f98914d6781d05, State: Initialized, Role: FOLLOWER
I20260811 02:51:04.177165 24785 consensus_queue.cc:260] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac88b0486d2430580f98914d6781d05" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 42039 } } peers { permanent_uuid: "252b32abb02f4e54a2abdabb7a9bee5f" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33109 } } peers { permanent_uuid: "b9410ac24fef470598e2c58f2588f7bf" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42011 } }
I20260811 02:51:04.177265 24787 consensus_queue.cc:260] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac88b0486d2430580f98914d6781d05" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 42039 } } peers { permanent_uuid: "252b32abb02f4e54a2abdabb7a9bee5f" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33109 } } peers { permanent_uuid: "b9410ac24fef470598e2c58f2588f7bf" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42011 } }
I20260811 02:51:04.177846 24538 heartbeater.cc:503] Master 127.21.236.190:42705 was elected leader, sending a full tablet report...
I20260811 02:51:04.178257 24785 ts_tablet_manager.cc:1428] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260811 02:51:04.178392 24787 ts_tablet_manager.cc:1428] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 02:51:04.178781 24759 heartbeater.cc:503] Master 127.21.236.190:42705 was elected leader, sending a full tablet report...
I20260811 02:51:04.180977 24789 raft_consensus.cc:348] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac88b0486d2430580f98914d6781d05" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 42039 } } peers { permanent_uuid: "252b32abb02f4e54a2abdabb7a9bee5f" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33109 } } peers { permanent_uuid: "b9410ac24fef470598e2c58f2588f7bf" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42011 } }
I20260811 02:51:04.181255 24789 raft_consensus.cc:374] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:04.181310 24789 raft_consensus.cc:729] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9410ac24fef470598e2c58f2588f7bf, State: Initialized, Role: FOLLOWER
I20260811 02:51:04.181778 24789 consensus_queue.cc:260] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac88b0486d2430580f98914d6781d05" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 42039 } } peers { permanent_uuid: "252b32abb02f4e54a2abdabb7a9bee5f" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33109 } } peers { permanent_uuid: "b9410ac24fef470598e2c58f2588f7bf" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42011 } }
I20260811 02:51:04.182456 24318 heartbeater.cc:503] Master 127.21.236.190:42705 was elected leader, sending a full tablet report...
I20260811 02:51:04.183054 24789 ts_tablet_manager.cc:1428] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260811 02:51:04.219587 24321 tablet.cc:2369] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:51:04.359711 24760 tablet.cc:2369] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 02:51:04.414829 24539 tablet.cc:2369] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 02:51:04.451648 24798 raft_consensus.cc:482] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 02:51:04.451900 24798 raft_consensus.cc:504] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac88b0486d2430580f98914d6781d05" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 42039 } } peers { permanent_uuid: "252b32abb02f4e54a2abdabb7a9bee5f" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33109 } } peers { permanent_uuid: "b9410ac24fef470598e2c58f2588f7bf" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42011 } }
I20260811 02:51:04.453169 24798 leader_election.cc:288] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 252b32abb02f4e54a2abdabb7a9bee5f (127.21.236.130:33109), b9410ac24fef470598e2c58f2588f7bf (127.21.236.129:42011)
I20260811 02:51:04.458101 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.453518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:49593 (local address 127.21.236.130:33109)
0811 02:51:04.453675 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:04.453680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:04.453699 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:51:04.453764 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:04.453769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:04.453829 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:04.453923 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:04.454376 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.455190 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.456829 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.457420 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.457546 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:04.457596 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:04.457887 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 02:51:04.457971 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 02:51:04.458101 24762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.453383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.130:33109 (local address 127.21.236.131:49593)
0811 02:51:04.453575 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:04.453591 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:04.453681 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:04.454080 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:04.454087 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:04.454109 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:04.454234 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:04.454240 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.455315 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.455321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:04.456694 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:04.456706 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.457566 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.457570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:04.457685 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:04.457761 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 02:51:04.457872 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 02:51:04.457954 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":24}
I20260811 02:51:04.458209 24783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.454010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.236.131:47921 (local address 127.21.236.129:42011)
0811 02:51:04.454128 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:04.454133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:04.454151 (+ 18us) server_negotiation.cc:407] Connection header received
0811 02:51:04.454206 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:04.454209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:04.454253 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:04.454328 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 02:51:04.454677 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.455457 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.456993 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.457436 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.457583 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:04.457632 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:04.458001 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 02:51:04.458082 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:51:04.458541 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.453444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42011 (local address 127.21.236.131:47921)
0811 02:51:04.453941 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:04.453957 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:04.454057 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:04.454410 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:04.454413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:04.454431 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 02:51:04.454549 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:04.454555 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.455595 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.455599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:04.456850 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:04.456872 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.457695 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.457699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:04.457845 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:04.457868 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 02:51:04.458338 (+ 470us) client_negotiation.cc:240] Negotiation successful
0811 02:51:04.458412 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":23,"thread_start_us":149,"threads_started":1}
I20260811 02:51:04.458868 24464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73c36745ec544c8aba0c9becb65f2515" candidate_uuid: "eac88b0486d2430580f98914d6781d05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "252b32abb02f4e54a2abdabb7a9bee5f" is_pre_election: true
I20260811 02:51:04.459262 24464 raft_consensus.cc:2454] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac88b0486d2430580f98914d6781d05 in term 0.
I20260811 02:51:04.459563 24245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73c36745ec544c8aba0c9becb65f2515" candidate_uuid: "eac88b0486d2430580f98914d6781d05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9410ac24fef470598e2c58f2588f7bf" is_pre_election: true
I20260811 02:51:04.459821 24245 raft_consensus.cc:2454] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac88b0486d2430580f98914d6781d05 in term 0.
I20260811 02:51:04.459844 24572 leader_election.cc:302] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 252b32abb02f4e54a2abdabb7a9bee5f, eac88b0486d2430580f98914d6781d05; no voters:
I20260811 02:51:04.460215 24798 raft_consensus.cc:2781] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 02:51:04.460330 24798 raft_consensus.cc:482] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 02:51:04.460391 24798 raft_consensus.cc:3037] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:04.461685 24798 raft_consensus.cc:504] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac88b0486d2430580f98914d6781d05" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 42039 } } peers { permanent_uuid: "252b32abb02f4e54a2abdabb7a9bee5f" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33109 } } peers { permanent_uuid: "b9410ac24fef470598e2c58f2588f7bf" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42011 } }
I20260811 02:51:04.462128 24798 leader_election.cc:288] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [CANDIDATE]: Term 1 election: Requested vote from peers 252b32abb02f4e54a2abdabb7a9bee5f (127.21.236.130:33109), b9410ac24fef470598e2c58f2588f7bf (127.21.236.129:42011)
I20260811 02:51:04.462498 24245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73c36745ec544c8aba0c9becb65f2515" candidate_uuid: "eac88b0486d2430580f98914d6781d05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9410ac24fef470598e2c58f2588f7bf"
I20260811 02:51:04.462491 24464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73c36745ec544c8aba0c9becb65f2515" candidate_uuid: "eac88b0486d2430580f98914d6781d05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "252b32abb02f4e54a2abdabb7a9bee5f"
I20260811 02:51:04.462623 24245 raft_consensus.cc:3037] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:04.462657 24464 raft_consensus.cc:3037] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:04.463850 24464 raft_consensus.cc:2454] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac88b0486d2430580f98914d6781d05 in term 1.
I20260811 02:51:04.463959 24245 raft_consensus.cc:2454] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac88b0486d2430580f98914d6781d05 in term 1.
I20260811 02:51:04.464231 24572 leader_election.cc:302] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 252b32abb02f4e54a2abdabb7a9bee5f, eac88b0486d2430580f98914d6781d05; no voters:
I20260811 02:51:04.464423 24798 raft_consensus.cc:2781] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:04.464687 24798 raft_consensus.cc:686] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [term 1 LEADER]: Becoming Leader. State: Replica: eac88b0486d2430580f98914d6781d05, State: Running, Role: LEADER
I20260811 02:51:04.465036 24798 consensus_queue.cc:237] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac88b0486d2430580f98914d6781d05" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 42039 } } peers { permanent_uuid: "252b32abb02f4e54a2abdabb7a9bee5f" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33109 } } peers { permanent_uuid: "b9410ac24fef470598e2c58f2588f7bf" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42011 } }
I20260811 02:51:04.468320 23991 catalog_manager.cc:5495] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 reported cstate change: term changed from 0 to 1, leader changed from <none> to eac88b0486d2430580f98914d6781d05 (127.21.236.131). New cstate: current_term: 1 leader_uuid: "eac88b0486d2430580f98914d6781d05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac88b0486d2430580f98914d6781d05" member_type: VOTER last_known_addr { host: "127.21.236.131" port: 42039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "252b32abb02f4e54a2abdabb7a9bee5f" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9410ac24fef470598e2c58f2588f7bf" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42011 } health_report { overall_health: UNKNOWN } } }
I20260811 02:51:04.518298 24779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.514586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48580 (local address 127.21.236.131:42039)
0811 02:51:04.514748 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:04.514753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:04.514768 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:51:04.514970 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:04.514973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:04.515032 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:04.515171 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:04.515421 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.515925 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.516831 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:04.517058 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:04.517172 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:04.517490 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:04.517905 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:04.518055 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:04.518096 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 02:51:04.518149 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 02:51:04.524360 24464 raft_consensus.cc:1261] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f [term 1 FOLLOWER]: Refusing update from remote peer eac88b0486d2430580f98914d6781d05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:04.524360 24245 raft_consensus.cc:1261] T 73c36745ec544c8aba0c9becb65f2515 P b9410ac24fef470598e2c58f2588f7bf [term 1 FOLLOWER]: Refusing update from remote peer eac88b0486d2430580f98914d6781d05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:04.525038 24798 consensus_queue.cc:1035] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9410ac24fef470598e2c58f2588f7bf" member_type: VOTER last_known_addr { host: "127.21.236.129" port: 42011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:04.525249 24808 consensus_queue.cc:1035] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "252b32abb02f4e54a2abdabb7a9bee5f" member_type: VOTER last_known_addr { host: "127.21.236.130" port: 33109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:04.533337 24817 mvcc.cc:205] Tried to move back new op lower bound from 7317162657883762688 to 7317162657651396608. Current Snapshot: MvccSnapshot[applied={T|T < 7317162657883762688}]
I20260811 02:51:04.533524 24823 mvcc.cc:205] Tried to move back new op lower bound from 7317162657883762688 to 7317162657651396608. Current Snapshot: MvccSnapshot[applied={T|T < 7317162657883762688}]
I20260811 02:51:04.540978 23990 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:51:04.541159 23990 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 02:51:04.543478 23990 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afe12713a1df43f48bb428bb2bbc03ae: Sending DeleteTablet for 3 replicas of tablet 73c36745ec544c8aba0c9becb65f2515
I20260811 02:51:04.544354 24198 tablet_service.cc:1514] Processing DeleteTablet for tablet 73c36745ec544c8aba0c9becb65f2515 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:04 UTC) from {username='slave'} at 127.0.0.1:36578
I20260811 02:51:04.544430 24653 tablet_service.cc:1514] Processing DeleteTablet for tablet 73c36745ec544c8aba0c9becb65f2515 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:04 UTC) from {username='slave'} at 127.0.0.1:48564
I20260811 02:51:04.544864 24427 tablet_service.cc:1514] Processing DeleteTablet for tablet 73c36745ec544c8aba0c9becb65f2515 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 02:51:04 UTC) from {username='slave'} at 127.0.0.1:37516
I20260811 02:51:04.544886 24837 tablet_replica.cc:330] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05: stopping tablet replica
I20260811 02:51:04.545260 24838 tablet_replica.cc:330] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f: stopping tablet replica
I20260811 02:51:04.545378 24837 raft_consensus.cc:2229] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [term 1 LEADER]: Raft consensus shutting down.
I20260811 02:51:04.545539 24838 raft_consensus.cc:2229] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 02:51:04.545847 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 24089
I20260811 02:51:04.545881 24838 raft_consensus.cc:2258] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:51:04.545851 24837 raft_consensus.cc:2258] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 02:51:04.550436 24838 ts_tablet_manager.cc:1905] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:04.553596 24837 ts_tablet_manager.cc:1905] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 02:51:04.554296 24838 ts_tablet_manager.cc:1918] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:51:04.554435 24838 log.cc:1196] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-1/wal/wals/73c36745ec544c8aba0c9becb65f2515
I20260811 02:51:04.554850 24838 ts_tablet_manager.cc:1939] T 73c36745ec544c8aba0c9becb65f2515 P 252b32abb02f4e54a2abdabb7a9bee5f: Deleting consensus metadata
I20260811 02:51:04.556043 23964 catalog_manager.cc:4841] TS 252b32abb02f4e54a2abdabb7a9bee5f (127.21.236.130:33109): tablet 73c36745ec544c8aba0c9becb65f2515 (table test-table [id=8420a7a0dab14d5cb3161704723df003]) successfully deleted
I20260811 02:51:04.556514 24837 ts_tablet_manager.cc:1918] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 02:51:04.556617 24837 log.cc:1196] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05: Deleting WAL directory at /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786416395682572-22450-0/minicluster-data/ts-2/wal/wals/73c36745ec544c8aba0c9becb65f2515
I20260811 02:51:04.556942 24837 ts_tablet_manager.cc:1939] T 73c36745ec544c8aba0c9becb65f2515 P eac88b0486d2430580f98914d6781d05: Deleting consensus metadata
I20260811 02:51:04.557981 23964 catalog_manager.cc:4841] TS eac88b0486d2430580f98914d6781d05 (127.21.236.131:42039): tablet 73c36745ec544c8aba0c9becb65f2515 (table test-table [id=8420a7a0dab14d5cb3161704723df003]) successfully deleted
W20260811 02:51:04.565063 23962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.236.129:42011 (error 108)
I20260811 02:51:04.566004 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 24326
I20260811 02:51:04.566186 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:04.565838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.129:42011 (local address 127.0.0.1:36590)
0811 02:51:04.566019 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:04.566096 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.129:42011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260811 02:51:04.575645 23962 catalog_manager.cc:4584] TS b9410ac24fef470598e2c58f2588f7bf (127.21.236.129:42011): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 73c36745ec544c8aba0c9becb65f2515: Network error: Client connection negotiation failed: client connection to 127.21.236.129:42011: connect: Connection refused (error 111)
I20260811 02:51:04.582373 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 24544
I20260811 02:51:04.593919 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 23939
2026-08-11T02:51:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1383 ms)
[----------] 6 tests from EncryptionPolicyTest (11997 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:04 dist-test-slave-n326 krb5kdc[24842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:04 dist-test-slave-n326 krb5kdc[24842](info): set up 2 sockets
Aug 11 02:51:04 dist-test-slave-n326 krb5kdc[24842](info): commencing operation
krb5kdc: starting...
W20260811 02:51:05.937741 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.317s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:06 dist-test-slave-n326 krb5kdc[24842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:51:06.091372 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33189
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:06.208612 24861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:06.208856 24861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:06.208932 24861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:06.212915 24861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:06.212975 24861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:06.212996 24861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:06.213012 24861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:06.213066 24861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:06.218214 24861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33189
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:06.219280 24861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:06.220319 24861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:51:06.229175 24861 server_base.cc:1034] running on GCE node
W20260811 02:51:06.229007 24868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:06.229028 24871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:06.229034 24869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:06.229977 24861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:06.231223 24861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:06.232405 24861 hybrid_clock.cc:648] HybridClock initialized: now 1786416666232382 us; error 34 us; skew 500 ppm
Aug 11 02:51:06 dist-test-slave-n326 krb5kdc[24842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416666, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:06.235523 24861 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:51:06.236418 24861 webserver.cc:460] Webserver started at http://127.0.0.1:32949/ using document root <none> and password file <none>
I20260811 02:51:06.236982 24861 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:06.237047 24861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:06.237229 24861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:06.239550 24861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "1a7847bdc2bb44f9afeeff9c19c4c093"
format_stamp: "Formatted at 2026-08-11 02:51:06 on dist-test-slave-n326"
server_key: "b989d3554f71fe198a249307b0444435"
server_key_iv: "9681b3b7269e3f7dbdf3251ee63fe43e"
server_key_version: "encryptionkey@0"
I20260811 02:51:06.240106 24861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "1a7847bdc2bb44f9afeeff9c19c4c093"
format_stamp: "Formatted at 2026-08-11 02:51:06 on dist-test-slave-n326"
server_key: "b989d3554f71fe198a249307b0444435"
server_key_iv: "9681b3b7269e3f7dbdf3251ee63fe43e"
server_key_version: "encryptionkey@0"
I20260811 02:51:06.244064 24861 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 02:51:06.246600 24885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:06.247977 24861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:51:06.248150 24861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "1a7847bdc2bb44f9afeeff9c19c4c093"
format_stamp: "Formatted at 2026-08-11 02:51:06 on dist-test-slave-n326"
server_key: "b989d3554f71fe198a249307b0444435"
server_key_iv: "9681b3b7269e3f7dbdf3251ee63fe43e"
server_key_version: "encryptionkey@0"
I20260811 02:51:06.248276 24861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:06.265432 24861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:06.268536 24861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:06.268757 24861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:06.276634 24861 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33189
I20260811 02:51:06.276657 24973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33189 every 8 connection(s)
I20260811 02:51:06.277667 24861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:06.278836 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 24861
I20260811 02:51:06.278966 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:06.279330 22450 external_mini_cluster.cc:1428] Setting key 93a3f97f655bd433a00eb92d9a6e6e1f
I20260811 02:51:06.281255 24975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:06.288178 24975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093: Bootstrap starting.
Aug 11 02:51:06 dist-test-slave-n326 krb5kdc[24842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 02:51:06.290768 24975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:06.291828 24975 log.cc:824] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:06.294310 24975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093: No bootstrap required, opened a new log
I20260811 02:51:06.295894 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:06.280856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58731 (local address 127.0.0.1:33189)
0811 02:51:06.281327 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:06.281348 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:06.281404 (+ 56us) server_negotiation.cc:407] Connection header received
0811 02:51:06.282250 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:06.282274 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:06.282699 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:06.283083 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:06.283926 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:06.285433 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:06.286409 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:06.287163 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:06.287340 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:06.290297 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:06.290304 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:06.290319 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:06.290349 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:06.292422 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:06.293147 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:06.293154 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:06.293159 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:06.293296 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:06.293684 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:06.293689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:06.293691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:06.294179 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:06.294392 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:06.294732 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 02:51:06.294956 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":135,"threads_started":1}
I20260811 02:51:06.298700 24975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7847bdc2bb44f9afeeff9c19c4c093" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } }
I20260811 02:51:06.298945 24975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:06.299013 24975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a7847bdc2bb44f9afeeff9c19c4c093, State: Initialized, Role: FOLLOWER
I20260811 02:51:06.299168 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:06.299492 24975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7847bdc2bb44f9afeeff9c19c4c093" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } }
I20260811 02:51:06.299664 24975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:06.299743 24975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:06.299865 24975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:06.300838 24975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7847bdc2bb44f9afeeff9c19c4c093" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } }
I20260811 02:51:06.301175 24975 leader_election.cc:302] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a7847bdc2bb44f9afeeff9c19c4c093; no voters:
I20260811 02:51:06.301522 24975 leader_election.cc:288] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:06.301604 24991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:06.301966 24991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [term 1 LEADER]: Becoming Leader. State: Replica: 1a7847bdc2bb44f9afeeff9c19c4c093, State: Running, Role: LEADER
I20260811 02:51:06.302312 24991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7847bdc2bb44f9afeeff9c19c4c093" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } }
I20260811 02:51:06.302886 24975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:06.304003 24996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a7847bdc2bb44f9afeeff9c19c4c093" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7847bdc2bb44f9afeeff9c19c4c093" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } } }
I20260811 02:51:06.304494 24996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:06.304371 24998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a7847bdc2bb44f9afeeff9c19c4c093. Latest consensus state: current_term: 1 leader_uuid: "1a7847bdc2bb44f9afeeff9c19c4c093" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a7847bdc2bb44f9afeeff9c19c4c093" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } } }
I20260811 02:51:06.304895 24998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093 [sys.catalog]: This master's current role is: LEADER
W20260811 02:51:06.308815 25026 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:51:06.308902 25026 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:51:06.308986 25024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:06.310081 25024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:06.313369 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:06.301689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55151 (local address 127.0.0.1:33189)
0811 02:51:06.301818 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:06.301826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:06.303349 (+ 1523us) server_negotiation.cc:407] Connection header received
0811 02:51:06.303539 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:06.303544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:06.303597 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:06.303671 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:06.304825 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:06.305669 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:06.306932 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:06.307285 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:06.307396 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:06.309205 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:06.309209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:06.309212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:06.309240 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:06.311097 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:06.311742 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:06.311746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:06.311748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:06.311837 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:06.312261 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:06.312266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:06.312268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:06.312484 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:06.312664 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:06.312966 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 02:51:06.313071 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 02:51:06.316258 25024 catalog_manager.cc:1347] Generated new cluster ID: c5edacb002c1410d88ae035f1f5e2c50
I20260811 02:51:06.316331 25024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:06.330269 25024 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:06.331331 25024 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:06.342321 25024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a7847bdc2bb44f9afeeff9c19c4c093: Generated new TSK 0
I20260811 02:51:06.343051 25024 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:06.353252 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 24861
W20260811 02:51:06.369997 24840 connection.cc:535] client connection to 127.0.0.1:33189 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33189): Connection reset by peer (error 104)
2026-08-11T02:51:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1763 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:06 dist-test-slave-n326 krb5kdc[25041](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:06 dist-test-slave-n326 krb5kdc[25041](info): set up 2 sockets
Aug 11 02:51:06 dist-test-slave-n326 krb5kdc[25041](info): commencing operation
krb5kdc: starting...
W20260811 02:51:07.680778 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:07 dist-test-slave-n326 krb5kdc[25041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:51:07.822875 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40627
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:07.942219 25067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:07.942538 25067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:07.942631 25067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:07.946663 25067 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:07.946765 25067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:07.946807 25067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:07.946866 25067 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:07.946907 25067 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:07.952291 25067 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40627
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:07.953368 25067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:07.954402 25067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:07.963476 25085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:07.963584 25079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:07.963547 25067 server_base.cc:1034] running on GCE node
W20260811 02:51:07.963654 25080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:07.964398 25067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:07.965597 25067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:07.966813 25067 hybrid_clock.cc:648] HybridClock initialized: now 1786416667966793 us; error 41 us; skew 500 ppm
Aug 11 02:51:07 dist-test-slave-n326 krb5kdc[25041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416667, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:07.969866 25067 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:51:07.970810 25067 webserver.cc:460] Webserver started at http://127.0.0.1:39549/ using document root <none> and password file <none>
I20260811 02:51:07.971486 25067 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:07.971592 25067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:07.971850 25067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:07.974572 25067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "5a64f93000254503900e8ab315d3f86d"
format_stamp: "Formatted at 2026-08-11 02:51:07 on dist-test-slave-n326"
server_key: "af1e0767ea8c24e08d77a3bfd37460e6"
server_key_iv: "570d404587653a24582778123fa75652"
server_key_version: "encryptionkey@0"
I20260811 02:51:07.975261 25067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "5a64f93000254503900e8ab315d3f86d"
format_stamp: "Formatted at 2026-08-11 02:51:07 on dist-test-slave-n326"
server_key: "af1e0767ea8c24e08d77a3bfd37460e6"
server_key_iv: "570d404587653a24582778123fa75652"
server_key_version: "encryptionkey@0"
I20260811 02:51:07.979646 25067 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:51:07.982326 25099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:07.983762 25067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:07.983932 25067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "5a64f93000254503900e8ab315d3f86d"
format_stamp: "Formatted at 2026-08-11 02:51:07 on dist-test-slave-n326"
server_key: "af1e0767ea8c24e08d77a3bfd37460e6"
server_key_iv: "570d404587653a24582778123fa75652"
server_key_version: "encryptionkey@0"
I20260811 02:51:07.984058 25067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:08.005064 25067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:08.008832 25067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:08.009109 25067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:08.017037 25067 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40627
I20260811 02:51:08.017073 25190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40627 every 8 connection(s)
I20260811 02:51:08.018105 25067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:08.020630 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 25067
I20260811 02:51:08.020787 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:08.021032 25191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:08.021114 22450 external_mini_cluster.cc:1428] Setting key 85342d4dc0a60ecaa75d8995f95e4acc
I20260811 02:51:08.028174 25191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d: Bootstrap starting.
I20260811 02:51:08.030690 25191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:08.030936 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:08.022548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44187 (local address 127.0.0.1:40627)
0811 02:51:08.022993 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:08.023010 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:08.023049 (+ 39us) server_negotiation.cc:407] Connection header received
0811 02:51:08.023916 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:08.023936 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:08.024305 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:08.024599 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:08.025007 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:08.026124 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:08.027383 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:08.027882 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:08.028020 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:08.029059 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:08.029064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:08.029078 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:08.029483 (+ 405us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:08.029551 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:08.029577 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:08.029695 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:08.029903 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 02:51:08.030103 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":150,"threads_started":1}
I20260811 02:51:08.031535 25191 log.cc:824] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:08.033869 25191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d: No bootstrap required, opened a new log
I20260811 02:51:08.034072 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:08.038966 25191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a64f93000254503900e8ab315d3f86d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40627 } }
I20260811 02:51:08.039314 25191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:08.039397 25191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a64f93000254503900e8ab315d3f86d, State: Initialized, Role: FOLLOWER
I20260811 02:51:08.039945 25191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a64f93000254503900e8ab315d3f86d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40627 } }
I20260811 02:51:08.040158 25191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:08.040244 25191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:08.040354 25191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:08.040892 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:08.036564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52721 (local address 127.0.0.1:40627)
0811 02:51:08.036719 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:08.036725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:08.036808 (+ 83us) server_negotiation.cc:407] Connection header received
0811 02:51:08.036905 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:08.036908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:08.036954 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:08.037048 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:08.037290 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:08.038139 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:08.038962 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:08.039352 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:08.039470 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:08.040411 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:08.040416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:08.040419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:08.040436 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:08.040469 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:08.040495 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:08.040608 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:08.040717 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 02:51:08.040752 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 02:51:08.041623 25191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a64f93000254503900e8ab315d3f86d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40627 } }
I20260811 02:51:08.042042 25191 leader_election.cc:302] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a64f93000254503900e8ab315d3f86d; no voters:
I20260811 02:51:08.042346 25191 leader_election.cc:288] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:08.042464 25204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:08.042722 25204 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [term 1 LEADER]: Becoming Leader. State: Replica: 5a64f93000254503900e8ab315d3f86d, State: Running, Role: LEADER
I20260811 02:51:08.043043 25204 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a64f93000254503900e8ab315d3f86d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40627 } }
I20260811 02:51:08.043829 25191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:08.044895 25205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a64f93000254503900e8ab315d3f86d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a64f93000254503900e8ab315d3f86d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40627 } } }
I20260811 02:51:08.045044 25205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:08.045446 25221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:08.045413 25208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a64f93000254503900e8ab315d3f86d. Latest consensus state: current_term: 1 leader_uuid: "5a64f93000254503900e8ab315d3f86d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a64f93000254503900e8ab315d3f86d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40627 } } }
I20260811 02:51:08.045665 25208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:08.048571 25221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:08.054141 25221 catalog_manager.cc:1347] Generated new cluster ID: 43352705440149db88f2dbc1b28f61cf
I20260811 02:51:08.054219 25221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:08.092141 25221 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:08.093411 25221 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:08.102074 25221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a64f93000254503900e8ab315d3f86d: Generated new TSK 0
I20260811 02:51:08.102813 25221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:08.118541 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 25067
W20260811 02:51:08.130132 25038 connection.cc:535] client connection to 127.0.0.1:40627 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40627): Connection reset by peer (error 104)
2026-08-11T02:51:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1760 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:08 dist-test-slave-n326 krb5kdc[25237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:08 dist-test-slave-n326 krb5kdc[25237](info): set up 2 sockets
Aug 11 02:51:08 dist-test-slave-n326 krb5kdc[25237](info): commencing operation
krb5kdc: starting...
W20260811 02:51:09.472005 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:09 dist-test-slave-n326 krb5kdc[25237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:51:09.613978 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41941
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:09.729347 25261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:09.729693 25261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:09.729779 25261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:09.733932 25261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:09.734028 25261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:09.734067 25261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:09.734129 25261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:09.734169 25261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:09.739642 25261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41941
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:09.740764 25261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:09.741829 25261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:09.750641 25269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:09.750721 25268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:09.750783 25261 server_base.cc:1034] running on GCE node
W20260811 02:51:09.750976 25271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:09.751590 25261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:09.752710 25261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:09.753952 25261 hybrid_clock.cc:648] HybridClock initialized: now 1786416669753925 us; error 45 us; skew 500 ppm
Aug 11 02:51:09 dist-test-slave-n326 krb5kdc[25237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416669, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:09.760394 25261 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:51:09.761304 25261 webserver.cc:460] Webserver started at http://127.0.0.1:38051/ using document root <none> and password file <none>
I20260811 02:51:09.761861 25261 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:09.761922 25261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:09.762107 25261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:09.764238 25261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "7f2f8f86c4d24cdea33b2de87287fc80"
format_stamp: "Formatted at 2026-08-11 02:51:09 on dist-test-slave-n326"
server_key: "e0dd628df355df72048bcb97fef0fec6"
server_key_iv: "9a64319d67ed15bcb8430abb4ef0dc81"
server_key_version: "encryptionkey@0"
I20260811 02:51:09.764760 25261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "7f2f8f86c4d24cdea33b2de87287fc80"
format_stamp: "Formatted at 2026-08-11 02:51:09 on dist-test-slave-n326"
server_key: "e0dd628df355df72048bcb97fef0fec6"
server_key_iv: "9a64319d67ed15bcb8430abb4ef0dc81"
server_key_version: "encryptionkey@0"
I20260811 02:51:09.768628 25261 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:51:09.771200 25282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:09.772511 25261 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:51:09.772655 25261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "7f2f8f86c4d24cdea33b2de87287fc80"
format_stamp: "Formatted at 2026-08-11 02:51:09 on dist-test-slave-n326"
server_key: "e0dd628df355df72048bcb97fef0fec6"
server_key_iv: "9a64319d67ed15bcb8430abb4ef0dc81"
server_key_version: "encryptionkey@0"
I20260811 02:51:09.772764 25261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:09.777590 25261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:09.780542 25261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:09.780750 25261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:09.788368 25261 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41941
I20260811 02:51:09.788427 25365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41941 every 8 connection(s)
I20260811 02:51:09.789427 25261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:09.790861 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 25261
I20260811 02:51:09.791023 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:09.791374 22450 external_mini_cluster.cc:1428] Setting key caf748a7d97ff5582ea1e1bdd4dad4ec
I20260811 02:51:09.792575 25367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:09.797056 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:09.792868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59779 (local address 127.0.0.1:41941)
0811 02:51:09.793274 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:09.793293 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:09.793334 (+ 41us) server_negotiation.cc:407] Connection header received
0811 02:51:09.794175 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:09.794199 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:09.794542 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:09.794907 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:09.795666 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:09.795672 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:09.795686 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:09.795700 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:09.795772 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:09.795800 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:09.795938 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:09.796005 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 02:51:09.796152 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260811 02:51:09.799220 25367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80: Bootstrap starting.
I20260811 02:51:09.799985 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:09.801872 25367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:09.802778 25367 log.cc:824] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:09.803992 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:09.802292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40913 (local address 127.0.0.1:41941)
0811 02:51:09.802432 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:09.802436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:09.802655 (+ 219us) server_negotiation.cc:407] Connection header received
0811 02:51:09.802777 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:09.802779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:09.802810 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:09.802895 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:09.803576 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:09.803579 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:09.803581 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:09.803592 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:09.803644 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:09.803661 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:09.803764 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:09.803835 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 02:51:09.803875 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:51:09.805157 25367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80: No bootstrap required, opened a new log
I20260811 02:51:09.809005 25367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f8f86c4d24cdea33b2de87287fc80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41941 } }
I20260811 02:51:09.809239 25367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:09.809278 25367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f2f8f86c4d24cdea33b2de87287fc80, State: Initialized, Role: FOLLOWER
I20260811 02:51:09.809649 25367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f8f86c4d24cdea33b2de87287fc80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41941 } }
I20260811 02:51:09.809762 25367 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:09.809811 25367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:09.809887 25367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:09.810767 25367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f8f86c4d24cdea33b2de87287fc80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41941 } }
I20260811 02:51:09.811134 25367 leader_election.cc:302] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f2f8f86c4d24cdea33b2de87287fc80; no voters:
I20260811 02:51:09.811491 25367 leader_election.cc:288] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:09.811609 25384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:09.811852 25384 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [term 1 LEADER]: Becoming Leader. State: Replica: 7f2f8f86c4d24cdea33b2de87287fc80, State: Running, Role: LEADER
I20260811 02:51:09.812180 25384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f8f86c4d24cdea33b2de87287fc80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41941 } }
I20260811 02:51:09.812815 25367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:09.813980 25385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f2f8f86c4d24cdea33b2de87287fc80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f8f86c4d24cdea33b2de87287fc80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41941 } } }
I20260811 02:51:09.814118 25385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:09.814486 25386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f2f8f86c4d24cdea33b2de87287fc80. Latest consensus state: current_term: 1 leader_uuid: "7f2f8f86c4d24cdea33b2de87287fc80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f2f8f86c4d24cdea33b2de87287fc80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41941 } } }
I20260811 02:51:09.815428 25397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:09.815784 25386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:09.818416 25397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:09.824010 25397 catalog_manager.cc:1347] Generated new cluster ID: d34f06e94a9b43ea962532315610673c
I20260811 02:51:09.824088 25397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:09.841188 25397 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:09.841558 25397 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:09.853838 25397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f2f8f86c4d24cdea33b2de87287fc80: Generated new TSK 0
I20260811 02:51:09.854657 25397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:09.859114 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 25261
2026-08-11T02:51:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1752 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:09 dist-test-slave-n326 krb5kdc[25417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:09 dist-test-slave-n326 krb5kdc[25417](info): set up 2 sockets
Aug 11 02:51:09 dist-test-slave-n326 krb5kdc[25417](info): commencing operation
krb5kdc: starting...
W20260811 02:51:11.260950 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.363s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:11 dist-test-slave-n326 krb5kdc[25417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:51:11.401669 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35519
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:11.514195 25438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:11.514519 25438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:11.514598 25438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:11.518608 25438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:11.518712 25438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:11.518749 25438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:11.518803 25438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:11.518854 25438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:11.524180 25438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:11.525308 25438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:11.526340 25438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:11.535058 25451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:11.535043 25455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:11.535046 25446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:11.535460 25438 server_base.cc:1034] running on GCE node
I20260811 02:51:11.536053 25438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:11.537225 25438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:11.538403 25438 hybrid_clock.cc:648] HybridClock initialized: now 1786416671538353 us; error 62 us; skew 500 ppm
Aug 11 02:51:11 dist-test-slave-n326 krb5kdc[25417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416671, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:11.541307 25438 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:51:11.542255 25438 webserver.cc:460] Webserver started at http://127.0.0.1:45333/ using document root <none> and password file <none>
I20260811 02:51:11.542836 25438 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:11.542932 25438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:11.543242 25438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:11.545308 25438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "375a031f4b404b7a9a90d7242dc16724"
format_stamp: "Formatted at 2026-08-11 02:51:11 on dist-test-slave-n326"
server_key: "cddad00473e2a63d0d7f2981345e25b1"
server_key_iv: "ff50fa1cbde878f6149a6e316433067a"
server_key_version: "encryptionkey@0"
I20260811 02:51:11.545881 25438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "375a031f4b404b7a9a90d7242dc16724"
format_stamp: "Formatted at 2026-08-11 02:51:11 on dist-test-slave-n326"
server_key: "cddad00473e2a63d0d7f2981345e25b1"
server_key_iv: "ff50fa1cbde878f6149a6e316433067a"
server_key_version: "encryptionkey@0"
I20260811 02:51:11.549671 25438 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 02:51:11.552053 25468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:11.553279 25438 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:51:11.553434 25438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "375a031f4b404b7a9a90d7242dc16724"
format_stamp: "Formatted at 2026-08-11 02:51:11 on dist-test-slave-n326"
server_key: "cddad00473e2a63d0d7f2981345e25b1"
server_key_iv: "ff50fa1cbde878f6149a6e316433067a"
server_key_version: "encryptionkey@0"
I20260811 02:51:11.553546 25438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:11.570014 25438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:11.573374 25438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:11.573608 25438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:11.581162 25438 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35519
I20260811 02:51:11.581197 25548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35519 every 8 connection(s)
I20260811 02:51:11.582161 25438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:11.584947 25552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:11.589063 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 25438
I20260811 02:51:11.589214 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:11.589524 22450 external_mini_cluster.cc:1428] Setting key e7f0fa2e59c88c17275503ab1e740f9b
I20260811 02:51:11.591250 25552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724: Bootstrap starting.
I20260811 02:51:11.593422 25552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:11.594373 25552 log.cc:824] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:11.596532 25552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724: No bootstrap required, opened a new log
Aug 11 02:51:11 dist-test-slave-n326 krb5kdc[25417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 02:51:11.600987 25552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375a031f4b404b7a9a90d7242dc16724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35519 } }
I20260811 02:51:11.601212 25552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:11.601246 25552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 375a031f4b404b7a9a90d7242dc16724, State: Initialized, Role: FOLLOWER
I20260811 02:51:11.601761 25552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375a031f4b404b7a9a90d7242dc16724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35519 } }
I20260811 02:51:11.601944 25552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:11.602026 25552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:11.602134 25552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:11.603291 25552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375a031f4b404b7a9a90d7242dc16724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35519 } }
I20260811 02:51:11.603682 25552 leader_election.cc:302] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 375a031f4b404b7a9a90d7242dc16724; no voters:
I20260811 02:51:11.604048 25552 leader_election.cc:288] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:11.604166 25557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:11.604423 25557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [term 1 LEADER]: Becoming Leader. State: Replica: 375a031f4b404b7a9a90d7242dc16724, State: Running, Role: LEADER
I20260811 02:51:11.604781 25557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375a031f4b404b7a9a90d7242dc16724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35519 } }
I20260811 02:51:11.605172 25555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:11.590970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45801 (local address 127.0.0.1:35519)
0811 02:51:11.591487 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:11.591503 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:11.591539 (+ 36us) server_negotiation.cc:407] Connection header received
0811 02:51:11.592166 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:11.592182 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:11.592504 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:11.592839 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:11.593676 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:11.595059 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:11.596011 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:11.596678 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:11.596836 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:11.599685 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:11.599690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:11.599702 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:11.599729 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:11.601681 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:11.602221 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:11.602228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:11.602235 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:11.602381 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:11.602709 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:11.602712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:11.602714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:11.603292 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:11.603490 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:11.603912 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 02:51:11.604240 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":140,"threads_started":1}
I20260811 02:51:11.605296 25552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:11.606437 25559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 375a031f4b404b7a9a90d7242dc16724. Latest consensus state: current_term: 1 leader_uuid: "375a031f4b404b7a9a90d7242dc16724" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375a031f4b404b7a9a90d7242dc16724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35519 } } }
I20260811 02:51:11.606585 25559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:11.606913 25558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "375a031f4b404b7a9a90d7242dc16724" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375a031f4b404b7a9a90d7242dc16724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35519 } } }
I20260811 02:51:11.606994 25558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:11.607000 25568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:11.608444 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:11.610483 25568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:11.616865 25568 catalog_manager.cc:1347] Generated new cluster ID: f6dd2a7725d74b7fbbaac2ce51f72937
I20260811 02:51:11.616931 25568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:11.620555 25555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:11.610716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38927 (local address 127.0.0.1:35519)
0811 02:51:11.610894 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:11.610898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:11.611506 (+ 608us) server_negotiation.cc:407] Connection header received
0811 02:51:11.611622 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:11.611626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:11.611675 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:11.611766 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:11.612622 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:11.613463 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:11.614642 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:11.615024 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:11.615211 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:11.617099 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:11.617103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:11.617106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:11.617132 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:11.618701 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:11.619215 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:11.619219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:11.619221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:11.619308 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:11.619625 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:11.619630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:11.619632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:11.619822 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:11.619975 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:11.620236 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 02:51:11.620378 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 02:51:11.636652 25568 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:11.637635 25568 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:11.643862 25568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 375a031f4b404b7a9a90d7242dc16724: Generated new TSK 0
I20260811 02:51:11.644675 25568 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:11.650400 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 25438
2026-08-11T02:51:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1785 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:11 dist-test-slave-n326 krb5kdc[25598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:11 dist-test-slave-n326 krb5kdc[25598](info): set up 2 sockets
Aug 11 02:51:11 dist-test-slave-n326 krb5kdc[25598](info): commencing operation
krb5kdc: starting...
W20260811 02:51:12.989789 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:13 dist-test-slave-n326 krb5kdc[25598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:51:13.126037 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33107
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:13.251330 25625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:13.251715 25625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:13.251827 25625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:13.256313 25625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:13.256420 25625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:13.256505 25625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:13.256559 25625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:13.256589 25625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:13.262020 25625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:13.263206 25625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:13.264240 25625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:13.273133 25638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:13.273191 25634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:13.273133 25633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:13.273411 25625 server_base.cc:1034] running on GCE node
I20260811 02:51:13.274277 25625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:13.275640 25625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:13.276846 25625 hybrid_clock.cc:648] HybridClock initialized: now 1786416673276818 us; error 45 us; skew 500 ppm
Aug 11 02:51:13 dist-test-slave-n326 krb5kdc[25598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416673, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:13.279882 25625 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:51:13.280836 25625 webserver.cc:460] Webserver started at http://127.0.0.1:34735/ using document root <none> and password file <none>
I20260811 02:51:13.281478 25625 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:13.281548 25625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:13.281847 25625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:13.284209 25625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "2c1a9da4898f48c6b151053fd4b90118"
format_stamp: "Formatted at 2026-08-11 02:51:13 on dist-test-slave-n326"
server_key: "18ab94026295800897d387bd8ac2eee5"
server_key_iv: "47603e7a0e5409fb9e328de09324e5e3"
server_key_version: "encryptionkey@0"
I20260811 02:51:13.284832 25625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "2c1a9da4898f48c6b151053fd4b90118"
format_stamp: "Formatted at 2026-08-11 02:51:13 on dist-test-slave-n326"
server_key: "18ab94026295800897d387bd8ac2eee5"
server_key_iv: "47603e7a0e5409fb9e328de09324e5e3"
server_key_version: "encryptionkey@0"
I20260811 02:51:13.289047 25625 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:51:13.291956 25652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:13.293349 25625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:13.293569 25625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "2c1a9da4898f48c6b151053fd4b90118"
format_stamp: "Formatted at 2026-08-11 02:51:13 on dist-test-slave-n326"
server_key: "18ab94026295800897d387bd8ac2eee5"
server_key_iv: "47603e7a0e5409fb9e328de09324e5e3"
server_key_version: "encryptionkey@0"
I20260811 02:51:13.293704 25625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:13.313537 25625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:13.316833 25625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:13.317077 25625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:13.325587 25625 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33107
I20260811 02:51:13.325613 25732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33107 every 8 connection(s)
I20260811 02:51:13.326665 25625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:13.329550 25734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:13.334084 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 25625
I20260811 02:51:13.334249 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:13.334589 22450 external_mini_cluster.cc:1428] Setting key 3281be2848bfaa22bdf9ad97a0e8c4cf
I20260811 02:51:13.335541 25734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118: Bootstrap starting.
I20260811 02:51:13.337937 25734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:13.338761 25734 log.cc:824] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:13.341012 25734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118: No bootstrap required, opened a new log
I20260811 02:51:13.344120 25740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:13.336059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58537 (local address 127.0.0.1:33107)
0811 02:51:13.336503 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:13.336524 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:13.336571 (+ 47us) server_negotiation.cc:407] Connection header received
0811 02:51:13.337262 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:13.337280 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:13.337644 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:13.337946 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:13.338372 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:13.339566 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:13.340761 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:13.341338 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:13.341492 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:13.342337 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:13.342342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:13.342355 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:13.342757 (+ 402us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:13.342812 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:13.342835 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:13.342932 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:13.343207 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 02:51:13.343419 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":144,"threads_started":1}
I20260811 02:51:13.344991 25734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1a9da4898f48c6b151053fd4b90118" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33107 } }
I20260811 02:51:13.345202 25734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:13.345273 25734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c1a9da4898f48c6b151053fd4b90118, State: Initialized, Role: FOLLOWER
I20260811 02:51:13.345717 25734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1a9da4898f48c6b151053fd4b90118" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33107 } }
I20260811 02:51:13.345873 25734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:13.346009 25734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:13.346127 25734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:13.347116 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:13.347137 25734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1a9da4898f48c6b151053fd4b90118" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33107 } }
I20260811 02:51:13.347488 25734 leader_election.cc:302] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c1a9da4898f48c6b151053fd4b90118; no voters:
I20260811 02:51:13.347843 25734 leader_election.cc:288] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:13.347939 25743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:13.348166 25743 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [term 1 LEADER]: Becoming Leader. State: Replica: 2c1a9da4898f48c6b151053fd4b90118, State: Running, Role: LEADER
I20260811 02:51:13.348519 25743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1a9da4898f48c6b151053fd4b90118" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33107 } }
I20260811 02:51:13.349208 25734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:13.349947 25746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c1a9da4898f48c6b151053fd4b90118. Latest consensus state: current_term: 1 leader_uuid: "2c1a9da4898f48c6b151053fd4b90118" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1a9da4898f48c6b151053fd4b90118" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33107 } } }
I20260811 02:51:13.350087 25746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:13.350733 25744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c1a9da4898f48c6b151053fd4b90118" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1a9da4898f48c6b151053fd4b90118" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33107 } } }
I20260811 02:51:13.350822 25744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:13.351222 25763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:13.354321 25763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:13.354456 25740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:13.349390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56555 (local address 127.0.0.1:33107)
0811 02:51:13.349513 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:13.349517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:13.349846 (+ 329us) server_negotiation.cc:407] Connection header received
0811 02:51:13.350054 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:13.350060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:13.350115 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:13.350178 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:13.350643 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:13.351461 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:13.352738 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:13.352974 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:13.353085 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:13.353945 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:13.353949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:13.353951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:13.353963 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:13.353998 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:13.354009 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:13.354091 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:13.354206 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 02:51:13.354255 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:51:13.361392 25763 catalog_manager.cc:1347] Generated new cluster ID: 0de8000f3fef4e818c8bd17b863e65d3
I20260811 02:51:13.361464 25763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:13.382989 25763 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:13.384379 25763 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:13.401407 25763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c1a9da4898f48c6b151053fd4b90118: Generated new TSK 0
I20260811 02:51:13.402245 25763 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:13.406538 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 25625
2026-08-11T02:51:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1751 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:13 dist-test-slave-n326 krb5kdc[25783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:13 dist-test-slave-n326 krb5kdc[25783](info): set up 2 sockets
Aug 11 02:51:13 dist-test-slave-n326 krb5kdc[25783](info): commencing operation
krb5kdc: starting...
W20260811 02:51:14.717792 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.283s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:14 dist-test-slave-n326 krb5kdc[25783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:51:14.854952 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36905
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:14.972576 25805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:14.972868 25805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:14.972913 25805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:14.977057 25805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:14.977130 25805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:14.977149 25805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:14.977166 25805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:14.977181 25805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:14.982854 25805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:14.984018 25805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:14.985023 25805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:14.993850 25813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:14.993860 25817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:14.994033 25814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:14.994131 25805 server_base.cc:1034] running on GCE node
I20260811 02:51:14.994676 25805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:14.995868 25805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:14.997062 25805 hybrid_clock.cc:648] HybridClock initialized: now 1786416674997027 us; error 45 us; skew 500 ppm
Aug 11 02:51:15 dist-test-slave-n326 krb5kdc[25783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416675, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:15.005858 25805 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:51:15.006924 25805 webserver.cc:460] Webserver started at http://127.0.0.1:36523/ using document root <none> and password file <none>
I20260811 02:51:15.007620 25805 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:15.007721 25805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:15.007984 25805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:15.010142 25805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "95c4898d016d4e6fb37bf018eb321c2b"
format_stamp: "Formatted at 2026-08-11 02:51:15 on dist-test-slave-n326"
server_key: "e953efd755cca6bd3d99f96fbe024824"
server_key_iv: "e79c64269159f367a400137ee5088bc0"
server_key_version: "encryptionkey@0"
I20260811 02:51:15.010727 25805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "95c4898d016d4e6fb37bf018eb321c2b"
format_stamp: "Formatted at 2026-08-11 02:51:15 on dist-test-slave-n326"
server_key: "e953efd755cca6bd3d99f96fbe024824"
server_key_iv: "e79c64269159f367a400137ee5088bc0"
server_key_version: "encryptionkey@0"
I20260811 02:51:15.014631 25805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 02:51:15.017174 25830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:15.018391 25805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 02:51:15.018535 25805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "95c4898d016d4e6fb37bf018eb321c2b"
format_stamp: "Formatted at 2026-08-11 02:51:15 on dist-test-slave-n326"
server_key: "e953efd755cca6bd3d99f96fbe024824"
server_key_iv: "e79c64269159f367a400137ee5088bc0"
server_key_version: "encryptionkey@0"
I20260811 02:51:15.018658 25805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:15.023360 25805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:15.026455 25805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:15.026674 25805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:15.034162 25805 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36905
I20260811 02:51:15.034195 25906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36905 every 8 connection(s)
I20260811 02:51:15.035225 25805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:15.037920 25907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:15.042889 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 25805
I20260811 02:51:15.043037 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:15.043358 22450 external_mini_cluster.cc:1428] Setting key c379c5fd7fe68c9717b3d3459428620e
I20260811 02:51:15.043656 25907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b: Bootstrap starting.
I20260811 02:51:15.046109 25907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:15.046911 25907 log.cc:824] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:15.048514 25912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:15.044727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54761 (local address 127.0.0.1:36905)
0811 02:51:15.045187 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:15.045201 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:15.045234 (+ 33us) server_negotiation.cc:407] Connection header received
0811 02:51:15.045864 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:15.045883 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:15.046153 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:15.046533 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:15.047344 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:15.047348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:15.047360 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:15.047370 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:15.047424 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:15.047443 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:15.047572 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:15.047691 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 02:51:15.047850 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":125,"threads_started":1}
I20260811 02:51:15.048949 25907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b: No bootstrap required, opened a new log
I20260811 02:51:15.051352 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:15.052775 25907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c4898d016d4e6fb37bf018eb321c2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } }
I20260811 02:51:15.053017 25907 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:15.053102 25907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95c4898d016d4e6fb37bf018eb321c2b, State: Initialized, Role: FOLLOWER
I20260811 02:51:15.053524 25907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c4898d016d4e6fb37bf018eb321c2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } }
I20260811 02:51:15.053725 25907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:15.053804 25907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:15.053927 25907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:15.055233 25907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c4898d016d4e6fb37bf018eb321c2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } }
I20260811 02:51:15.055339 25912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:15.053507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42145 (local address 127.0.0.1:36905)
0811 02:51:15.053655 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:15.053660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:15.053946 (+ 286us) server_negotiation.cc:407] Connection header received
0811 02:51:15.054101 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:15.054104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:15.054147 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:15.054233 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:15.054891 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:15.054895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:15.054897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:15.054909 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:15.054939 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:15.054951 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:15.055051 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:15.055143 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 02:51:15.055186 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 02:51:15.055585 25907 leader_election.cc:302] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95c4898d016d4e6fb37bf018eb321c2b; no voters:
I20260811 02:51:15.055908 25907 leader_election.cc:288] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:15.055958 25921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:15.056205 25921 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [term 1 LEADER]: Becoming Leader. State: Replica: 95c4898d016d4e6fb37bf018eb321c2b, State: Running, Role: LEADER
I20260811 02:51:15.056561 25921 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c4898d016d4e6fb37bf018eb321c2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } }
I20260811 02:51:15.057241 25907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:15.058257 25922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95c4898d016d4e6fb37bf018eb321c2b. Latest consensus state: current_term: 1 leader_uuid: "95c4898d016d4e6fb37bf018eb321c2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c4898d016d4e6fb37bf018eb321c2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } } }
I20260811 02:51:15.058396 25922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:15.058792 25941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:15.058765 25926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95c4898d016d4e6fb37bf018eb321c2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c4898d016d4e6fb37bf018eb321c2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36905 } } }
I20260811 02:51:15.059478 25926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:15.061833 25941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:15.066881 25941 catalog_manager.cc:1347] Generated new cluster ID: aad53d87f73d43939810fe69916fc599
I20260811 02:51:15.066948 25941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:15.086081 25941 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:15.086436 25941 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:15.097321 25941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95c4898d016d4e6fb37bf018eb321c2b: Generated new TSK 0
I20260811 02:51:15.098052 25941 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:15.109339 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 25805
2026-08-11T02:51:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1705 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:15 dist-test-slave-n326 krb5kdc[25974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:15 dist-test-slave-n326 krb5kdc[25974](info): set up 2 sockets
Aug 11 02:51:15 dist-test-slave-n326 krb5kdc[25974](info): commencing operation
krb5kdc: starting...
W20260811 02:51:16.527313 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.386s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:16 dist-test-slave-n326 krb5kdc[25974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:51:16.672669 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34883
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:16.796265 25992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:16.796624 25992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:16.796716 25992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:16.801156 25992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:16.801239 25992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:16.801277 25992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:16.801334 25992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:16.801400 25992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:16.806696 25992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:16.807796 25992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:16.808831 25992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:51:16.817406 25992 server_base.cc:1034] running on GCE node
W20260811 02:51:16.817397 26009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:16.817406 26002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:16.817399 26005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:16.818224 25992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:16.819366 25992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:16.820540 25992 hybrid_clock.cc:648] HybridClock initialized: now 1786416676820516 us; error 32 us; skew 500 ppm
Aug 11 02:51:16 dist-test-slave-n326 krb5kdc[25974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416676, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:16.823342 25992 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:51:16.824213 25992 webserver.cc:460] Webserver started at http://127.0.0.1:42573/ using document root <none> and password file <none>
I20260811 02:51:16.824800 25992 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:16.824894 25992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:16.825119 25992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:16.827288 25992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "46da50aad5034bf3b8434387e7521bfd"
format_stamp: "Formatted at 2026-08-11 02:51:16 on dist-test-slave-n326"
server_key: "0f9d1a576e56b8d84287005026d6ec8f"
server_key_iv: "d87df0c6ffd12f09674acb83e0ae8908"
server_key_version: "encryptionkey@0"
I20260811 02:51:16.827876 25992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "46da50aad5034bf3b8434387e7521bfd"
format_stamp: "Formatted at 2026-08-11 02:51:16 on dist-test-slave-n326"
server_key: "0f9d1a576e56b8d84287005026d6ec8f"
server_key_iv: "d87df0c6ffd12f09674acb83e0ae8908"
server_key_version: "encryptionkey@0"
I20260811 02:51:16.831734 25992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:51:16.834172 26024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:16.835444 25992 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:51:16.835583 25992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "46da50aad5034bf3b8434387e7521bfd"
format_stamp: "Formatted at 2026-08-11 02:51:16 on dist-test-slave-n326"
server_key: "0f9d1a576e56b8d84287005026d6ec8f"
server_key_iv: "d87df0c6ffd12f09674acb83e0ae8908"
server_key_version: "encryptionkey@0"
I20260811 02:51:16.835690 25992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:16.854241 25992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:16.857560 25992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:16.857808 25992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:16.865908 25992 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34883
I20260811 02:51:16.865954 26111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34883 every 8 connection(s)
I20260811 02:51:16.866952 25992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:16.869881 26112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:16.870792 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 25992
I20260811 02:51:16.870973 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:16.871340 22450 external_mini_cluster.cc:1428] Setting key 25b7307d447c92f268ad2a7a0cfcc6a5
I20260811 02:51:16.876839 26112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd: Bootstrap starting.
I20260811 02:51:16.879475 26112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:16.880262 26112 log.cc:824] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:51:16 dist-test-slave-n326 krb5kdc[25974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 02:51:16.882474 26112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd: No bootstrap required, opened a new log
I20260811 02:51:16.887126 26118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:16.872929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:59449 (local address 127.0.0.1:34883)
0811 02:51:16.873359 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:16.873378 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:16.873422 (+ 44us) server_negotiation.cc:407] Connection header received
0811 02:51:16.874251 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:16.874288 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:16.874711 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:16.875108 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:16.875846 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:16.877288 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:16.878263 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:16.878941 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:16.879130 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:16.881738 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:16.881744 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:16.881759 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:16.881791 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:16.883752 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:16.884259 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:16.884265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:16.884271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:16.884413 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:16.884756 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:16.884761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:16.884764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:16.885353 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:16.885565 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:16.885835 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 02:51:16.886078 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":108,"threads_started":1}
I20260811 02:51:16.887249 26112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46da50aad5034bf3b8434387e7521bfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } }
I20260811 02:51:16.887538 26112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:16.887593 26112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46da50aad5034bf3b8434387e7521bfd, State: Initialized, Role: FOLLOWER
I20260811 02:51:16.888058 26112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46da50aad5034bf3b8434387e7521bfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } }
I20260811 02:51:16.888229 26112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:16.888299 26112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:16.888406 26112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:16.889818 26112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46da50aad5034bf3b8434387e7521bfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } }
I20260811 02:51:16.890182 26112 leader_election.cc:302] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46da50aad5034bf3b8434387e7521bfd; no voters:
I20260811 02:51:16.890540 26112 leader_election.cc:288] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:16.890586 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:16.890631 26120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:16.890805 26120 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [term 1 LEADER]: Becoming Leader. State: Replica: 46da50aad5034bf3b8434387e7521bfd, State: Running, Role: LEADER
I20260811 02:51:16.891131 26120 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46da50aad5034bf3b8434387e7521bfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } }
I20260811 02:51:16.891865 26112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:16.892769 26121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46da50aad5034bf3b8434387e7521bfd. Latest consensus state: current_term: 1 leader_uuid: "46da50aad5034bf3b8434387e7521bfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46da50aad5034bf3b8434387e7521bfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } } }
I20260811 02:51:16.892927 26121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:16.893326 26139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:16.893738 26122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46da50aad5034bf3b8434387e7521bfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46da50aad5034bf3b8434387e7521bfd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } } }
I20260811 02:51:16.893846 26122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:16.896737 26139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:16.902824 26118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:16.893214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:42321 (local address 127.0.0.1:34883)
0811 02:51:16.893355 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:16.893360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:16.893375 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:51:16.893410 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:16.893413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:16.893461 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:16.893555 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:16.894427 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:16.895182 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:16.896251 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:16.896568 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:16.896666 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:16.898608 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:16.898612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:16.898615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:16.898643 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:16.900470 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:16.901263 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:16.901269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:16.901271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:16.901393 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:16.901814 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:16.901817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:16.901819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:16.902050 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:16.902205 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:16.902558 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 02:51:16.902664 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 02:51:16.903251 26139 catalog_manager.cc:1347] Generated new cluster ID: ddfaae8909bc4869a1d725495bd8b781
I20260811 02:51:16.903313 26139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:16.925523 26139 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:16.926455 26139 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:16.934615 26139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46da50aad5034bf3b8434387e7521bfd: Generated new TSK 0
I20260811 02:51:16.935400 26139 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:16.939992 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 25992
W20260811 02:51:16.954294 25966 connection.cc:535] client connection to 127.0.0.1:34883 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34883): Connection reset by peer (error 104)
2026-08-11T02:51:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1827 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:17 dist-test-slave-n326 krb5kdc[26159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:17 dist-test-slave-n326 krb5kdc[26159](info): set up 2 sockets
Aug 11 02:51:17 dist-test-slave-n326 krb5kdc[26159](info): commencing operation
krb5kdc: starting...
W20260811 02:51:18.298516 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.329s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:18 dist-test-slave-n326 krb5kdc[26159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:51:18.438079 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38163
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:18.550980 26184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:18.551321 26184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:18.551362 26184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:18.555276 26184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:18.555331 26184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:18.555351 26184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:18.555367 26184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:18.555382 26184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:18.561296 26184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:18.562418 26184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:18.563514 26184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:18.572836 26194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:18.572929 26184 server_base.cc:1034] running on GCE node
W20260811 02:51:18.572845 26192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:18.572857 26191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:18.573750 26184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:18.574851 26184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:18.576076 26184 hybrid_clock.cc:648] HybridClock initialized: now 1786416678576041 us; error 40 us; skew 500 ppm
Aug 11 02:51:18 dist-test-slave-n326 krb5kdc[26159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416678, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:18.579172 26184 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:51:18.580070 26184 webserver.cc:460] Webserver started at http://127.0.0.1:33771/ using document root <none> and password file <none>
I20260811 02:51:18.580624 26184 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:18.580689 26184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:18.580874 26184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:18.583012 26184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "6dbb349030024d37be49a8a42a661afa"
format_stamp: "Formatted at 2026-08-11 02:51:18 on dist-test-slave-n326"
server_key: "f3eb1d8fde37610c4fa5eb86279c7fb6"
server_key_iv: "7b93eefbcb6f6b546ffe8a8a42c5f94f"
server_key_version: "encryptionkey@0"
I20260811 02:51:18.583644 26184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "6dbb349030024d37be49a8a42a661afa"
format_stamp: "Formatted at 2026-08-11 02:51:18 on dist-test-slave-n326"
server_key: "f3eb1d8fde37610c4fa5eb86279c7fb6"
server_key_iv: "7b93eefbcb6f6b546ffe8a8a42c5f94f"
server_key_version: "encryptionkey@0"
I20260811 02:51:18.587610 26184 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:51:18.590178 26206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:18.591609 26184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:18.591727 26184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "6dbb349030024d37be49a8a42a661afa"
format_stamp: "Formatted at 2026-08-11 02:51:18 on dist-test-slave-n326"
server_key: "f3eb1d8fde37610c4fa5eb86279c7fb6"
server_key_iv: "7b93eefbcb6f6b546ffe8a8a42c5f94f"
server_key_version: "encryptionkey@0"
I20260811 02:51:18.591821 26184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:18.624392 26184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:18.627620 26184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:18.627801 26184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:18.635782 26184 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38163
I20260811 02:51:18.635814 26298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38163 every 8 connection(s)
I20260811 02:51:18.636826 26184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:18.639783 26301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:18.645543 26301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa: Bootstrap starting.
I20260811 02:51:18.645704 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 26184
I20260811 02:51:18.645828 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:18.646142 22450 external_mini_cluster.cc:1428] Setting key d9c137a5f41d4b26658fc1ac0db6559c
I20260811 02:51:18.648331 26301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:18.649084 26301 log.cc:824] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:18.651238 26301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa: No bootstrap required, opened a new log
I20260811 02:51:18.655539 26301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbb349030024d37be49a8a42a661afa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38163 } }
I20260811 02:51:18.655843 26301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:18.655934 26301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dbb349030024d37be49a8a42a661afa, State: Initialized, Role: FOLLOWER
I20260811 02:51:18.656322 26306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:18.647637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:50067 (local address 127.0.0.1:38163)
0811 02:51:18.648087 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:18.648103 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:18.648140 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:51:18.648915 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:18.648935 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:18.649349 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:18.649746 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:18.650183 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:18.651308 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:18.652570 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:18.653074 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:18.653258 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:18.654201 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:18.654206 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:18.654219 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:18.654692 (+ 473us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:18.654749 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:18.654773 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:18.654887 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:18.655246 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 02:51:18.655405 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":126,"threads_started":1}
I20260811 02:51:18.656450 26301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbb349030024d37be49a8a42a661afa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38163 } }
I20260811 02:51:18.656657 26301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:18.656736 26301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:18.656849 26301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:18.657883 26301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbb349030024d37be49a8a42a661afa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38163 } }
I20260811 02:51:18.658311 26301 leader_election.cc:302] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dbb349030024d37be49a8a42a661afa; no voters:
I20260811 02:51:18.658665 26301 leader_election.cc:288] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:18.658727 26312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:18.658998 26312 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [term 1 LEADER]: Becoming Leader. State: Replica: 6dbb349030024d37be49a8a42a661afa, State: Running, Role: LEADER
I20260811 02:51:18.659318 26312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbb349030024d37be49a8a42a661afa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38163 } }
I20260811 02:51:18.659583 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:18.660014 26301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:18.660640 26314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dbb349030024d37be49a8a42a661afa. Latest consensus state: current_term: 1 leader_uuid: "6dbb349030024d37be49a8a42a661afa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbb349030024d37be49a8a42a661afa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38163 } } }
I20260811 02:51:18.660772 26314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:18.661073 26313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dbb349030024d37be49a8a42a661afa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbb349030024d37be49a8a42a661afa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38163 } } }
I20260811 02:51:18.661165 26329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:18.661175 26313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:18.664700 26329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:18.667457 26306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:18.662021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:41817 (local address 127.0.0.1:38163)
0811 02:51:18.662115 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:18.662119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:18.662715 (+ 596us) server_negotiation.cc:407] Connection header received
0811 02:51:18.662897 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:18.662901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:18.662962 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:18.663011 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:18.663466 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:18.664220 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:18.665371 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:18.665720 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:18.665844 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:18.666867 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:18.666871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:18.666874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:18.666889 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:18.666922 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:18.666936 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:18.667038 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:18.667260 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 02:51:18.667302 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 02:51:18.670578 26329 catalog_manager.cc:1347] Generated new cluster ID: fde02d6ee9c04f8189b8e273629261d0
I20260811 02:51:18.670648 26329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:18.685592 26329 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:18.686581 26329 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:18.692333 26329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dbb349030024d37be49a8a42a661afa: Generated new TSK 0
I20260811 02:51:18.693050 26329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:18.703269 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 26184
W20260811 02:51:18.717531 26156 connection.cc:535] client connection to 127.0.0.1:38163 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38163): Connection reset by peer (error 104)
2026-08-11T02:51:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1763 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:18 dist-test-slave-n326 krb5kdc[26354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:18 dist-test-slave-n326 krb5kdc[26354](info): set up 2 sockets
Aug 11 02:51:18 dist-test-slave-n326 krb5kdc[26354](info): commencing operation
krb5kdc: starting...
W20260811 02:51:20.005645 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:20 dist-test-slave-n326 krb5kdc[26354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:51:20.134163 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33693
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:20.247712 26378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:20.248011 26378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:20.248083 26378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:20.252034 26378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:20.252118 26378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:20.252157 26378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:20.252215 26378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:20.252255 26378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:20.257373 26378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:20.258459 26378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:20.259560 26378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:20.268316 26403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:20.268311 26393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:20.268311 26395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:20.268674 26378 server_base.cc:1034] running on GCE node
I20260811 02:51:20.269282 26378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:20.270365 26378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:20.271555 26378 hybrid_clock.cc:648] HybridClock initialized: now 1786416680271539 us; error 28 us; skew 500 ppm
Aug 11 02:51:20 dist-test-slave-n326 krb5kdc[26354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416680, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:20.277719 26378 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:51:20.278617 26378 webserver.cc:460] Webserver started at http://127.0.0.1:46439/ using document root <none> and password file <none>
I20260811 02:51:20.279201 26378 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:20.279268 26378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:20.279441 26378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:20.281440 26378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "222ee4170cb44e15a8a80ea0b8be422f"
format_stamp: "Formatted at 2026-08-11 02:51:20 on dist-test-slave-n326"
server_key: "a79cc05ba57ac6b4f4aec77c2d3f463f"
server_key_iv: "226989bb660bba13ae6b0e3734f018db"
server_key_version: "encryptionkey@0"
I20260811 02:51:20.281956 26378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "222ee4170cb44e15a8a80ea0b8be422f"
format_stamp: "Formatted at 2026-08-11 02:51:20 on dist-test-slave-n326"
server_key: "a79cc05ba57ac6b4f4aec77c2d3f463f"
server_key_iv: "226989bb660bba13ae6b0e3734f018db"
server_key_version: "encryptionkey@0"
I20260811 02:51:20.285722 26378 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:51:20.288206 26416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:20.289522 26378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:20.289676 26378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "222ee4170cb44e15a8a80ea0b8be422f"
format_stamp: "Formatted at 2026-08-11 02:51:20 on dist-test-slave-n326"
server_key: "a79cc05ba57ac6b4f4aec77c2d3f463f"
server_key_iv: "226989bb660bba13ae6b0e3734f018db"
server_key_version: "encryptionkey@0"
I20260811 02:51:20.289798 26378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:20.294458 26378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:20.297770 26378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:20.297966 26378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:20.305505 26378 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33693
I20260811 02:51:20.305528 26512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33693 every 8 connection(s)
I20260811 02:51:20.306552 26378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:20.309481 26514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:20.310575 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 26378
I20260811 02:51:20.310730 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:20.311116 22450 external_mini_cluster.cc:1428] Setting key 8db6ea718f50ec9ede84ed5607156c15
I20260811 02:51:20.316485 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:20.312666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:36215 (local address 127.0.0.1:33693)
0811 02:51:20.313153 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:20.313169 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:20.313204 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:51:20.313831 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:20.313847 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:20.314130 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:20.314466 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:20.315317 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:20.315321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:20.315332 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:20.315343 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:20.315399 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:20.315417 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:20.315574 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:20.315644 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 02:51:20.315809 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":152,"threads_started":1}
I20260811 02:51:20.316793 26514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f: Bootstrap starting.
I20260811 02:51:20.319514 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:20.319710 26514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:20.320487 26514 log.cc:824] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:20.323163 26514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f: No bootstrap required, opened a new log
I20260811 02:51:20.323499 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:20.321607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:42073 (local address 127.0.0.1:33693)
0811 02:51:20.321720 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:20.321724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:20.321937 (+ 213us) server_negotiation.cc:407] Connection header received
0811 02:51:20.322081 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:20.322083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:20.322112 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:20.322170 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:20.322897 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:20.322899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:20.322901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:20.322910 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:20.322934 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:20.322944 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:20.323011 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:20.323340 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 02:51:20.323382 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:51:20.327209 26514 raft_consensus.cc:348] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "222ee4170cb44e15a8a80ea0b8be422f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } }
I20260811 02:51:20.327445 26514 raft_consensus.cc:374] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:20.327534 26514 raft_consensus.cc:729] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 222ee4170cb44e15a8a80ea0b8be422f, State: Initialized, Role: FOLLOWER
I20260811 02:51:20.327996 26514 consensus_queue.cc:260] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "222ee4170cb44e15a8a80ea0b8be422f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } }
I20260811 02:51:20.328148 26514 raft_consensus.cc:388] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:20.328226 26514 raft_consensus.cc:482] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:20.328341 26514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:20.329437 26514 raft_consensus.cc:504] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "222ee4170cb44e15a8a80ea0b8be422f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } }
I20260811 02:51:20.329798 26514 leader_election.cc:302] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 222ee4170cb44e15a8a80ea0b8be422f; no voters:
I20260811 02:51:20.330154 26514 leader_election.cc:288] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:20.330265 26536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:20.330509 26536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [term 1 LEADER]: Becoming Leader. State: Replica: 222ee4170cb44e15a8a80ea0b8be422f, State: Running, Role: LEADER
I20260811 02:51:20.330834 26536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "222ee4170cb44e15a8a80ea0b8be422f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } }
I20260811 02:51:20.331578 26514 sys_catalog.cc:564] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:20.332589 26538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 222ee4170cb44e15a8a80ea0b8be422f. Latest consensus state: current_term: 1 leader_uuid: "222ee4170cb44e15a8a80ea0b8be422f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "222ee4170cb44e15a8a80ea0b8be422f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } } }
I20260811 02:51:20.332767 26538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:20.333081 26537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "222ee4170cb44e15a8a80ea0b8be422f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "222ee4170cb44e15a8a80ea0b8be422f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } } }
I20260811 02:51:20.333182 26537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:20.333262 26550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:20.336290 26550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:20.342115 26550 catalog_manager.cc:1347] Generated new cluster ID: 3880d372cd38440e8b4f2626707c5c63
I20260811 02:51:20.342188 26550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:20.356083 26550 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:20.356459 26550 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:20.362632 26550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 222ee4170cb44e15a8a80ea0b8be422f: Generated new TSK 0
I20260811 02:51:20.363529 26550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:20.372439 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 26378
2026-08-11T02:51:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1677 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:20 dist-test-slave-n326 krb5kdc[26570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:20 dist-test-slave-n326 krb5kdc[26570](info): set up 2 sockets
Aug 11 02:51:20 dist-test-slave-n326 krb5kdc[26570](info): commencing operation
krb5kdc: starting...
W20260811 02:51:21.683523 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:21 dist-test-slave-n326 krb5kdc[26570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:51:21.818403 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37299
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:21.929366 26589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:21.929665 26589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:21.929745 26589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:21.933733 26589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:21.933833 26589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:21.933873 26589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:21.933930 26589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:21.933961 26589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:21.939386 26589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37299
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:21.940528 26589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:21.941603 26589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:21.950619 26606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:21.950690 26600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:21.950657 26604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:21.950975 26589 server_base.cc:1034] running on GCE node
I20260811 02:51:21.951673 26589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:21.952915 26589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:21.954136 26589 hybrid_clock.cc:648] HybridClock initialized: now 1786416681954101 us; error 46 us; skew 500 ppm
Aug 11 02:51:21 dist-test-slave-n326 krb5kdc[26570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:21.957397 26589 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:51:21.958303 26589 webserver.cc:460] Webserver started at http://127.0.0.1:36855/ using document root <none> and password file <none>
I20260811 02:51:21.958885 26589 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:21.958954 26589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:21.959209 26589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:21.961403 26589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "fc9144ef2a6c4f5b9cb72175d6add974"
format_stamp: "Formatted at 2026-08-11 02:51:21 on dist-test-slave-n326"
server_key: "4d1d59f7f143443f03f3eea94debf656"
server_key_iv: "55bdbb66c3ddbd98171004b641ab1f5e"
server_key_version: "encryptionkey@0"
I20260811 02:51:21.961994 26589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "fc9144ef2a6c4f5b9cb72175d6add974"
format_stamp: "Formatted at 2026-08-11 02:51:21 on dist-test-slave-n326"
server_key: "4d1d59f7f143443f03f3eea94debf656"
server_key_iv: "55bdbb66c3ddbd98171004b641ab1f5e"
server_key_version: "encryptionkey@0"
I20260811 02:51:21.966145 26589 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 02:51:21.968847 26616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:21.970391 26589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 02:51:21.970505 26589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "fc9144ef2a6c4f5b9cb72175d6add974"
format_stamp: "Formatted at 2026-08-11 02:51:21 on dist-test-slave-n326"
server_key: "4d1d59f7f143443f03f3eea94debf656"
server_key_iv: "55bdbb66c3ddbd98171004b641ab1f5e"
server_key_version: "encryptionkey@0"
I20260811 02:51:21.970607 26589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:21.985316 26589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:21.988713 26589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:21.988880 26589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:21.996622 26589 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37299
I20260811 02:51:21.996711 26705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37299 every 8 connection(s)
I20260811 02:51:21.997650 26589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:22.000720 26706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:22.005566 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 26589
I20260811 02:51:22.005716 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:22.006052 22450 external_mini_cluster.cc:1428] Setting key 673773dddb696e1529d9c48367c1dc7c
I20260811 02:51:22.007547 26706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974: Bootstrap starting.
I20260811 02:51:22.010076 26706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:22.010987 26706 log.cc:824] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:22.013118 26706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974: No bootstrap required, opened a new log
Aug 11 02:51:22 dist-test-slave-n326 krb5kdc[26570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 02:51:22.017536 26706 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc9144ef2a6c4f5b9cb72175d6add974" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37299 } }
I20260811 02:51:22.017805 26706 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:22.017900 26706 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc9144ef2a6c4f5b9cb72175d6add974, State: Initialized, Role: FOLLOWER
I20260811 02:51:22.018392 26706 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc9144ef2a6c4f5b9cb72175d6add974" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37299 } }
I20260811 02:51:22.018550 26706 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:22.018646 26706 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:22.018749 26706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:22.019752 26706 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc9144ef2a6c4f5b9cb72175d6add974" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37299 } }
I20260811 02:51:22.020102 26706 leader_election.cc:302] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc9144ef2a6c4f5b9cb72175d6add974; no voters:
I20260811 02:51:22.020467 26706 leader_election.cc:288] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:22.020588 26719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:22.020803 26719 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [term 1 LEADER]: Becoming Leader. State: Replica: fc9144ef2a6c4f5b9cb72175d6add974, State: Running, Role: LEADER
I20260811 02:51:22.021152 26719 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc9144ef2a6c4f5b9cb72175d6add974" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37299 } }
I20260811 02:51:22.021433 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:22.007678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:42137 (local address 127.0.0.1:37299)
0811 02:51:22.008149 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:22.008166 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:22.008203 (+ 37us) server_negotiation.cc:407] Connection header received
0811 02:51:22.008838 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:22.008854 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:22.009236 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:22.009570 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:22.010448 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:22.011576 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:22.012440 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:22.012929 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:22.013125 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:22.015992 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:22.015998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:22.016011 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:22.016040 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:22.017947 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:22.018521 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:22.018527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:22.018535 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:22.018734 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:22.019125 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:22.019129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:22.019131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:22.019528 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:22.019761 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:22.020089 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 02:51:22.020332 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":139,"threads_started":1}
I20260811 02:51:22.021888 26706 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:22.023272 26721 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc9144ef2a6c4f5b9cb72175d6add974. Latest consensus state: current_term: 1 leader_uuid: "fc9144ef2a6c4f5b9cb72175d6add974" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc9144ef2a6c4f5b9cb72175d6add974" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37299 } } }
I20260811 02:51:22.023397 26721 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:22.024227 26720 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc9144ef2a6c4f5b9cb72175d6add974" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc9144ef2a6c4f5b9cb72175d6add974" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37299 } } }
I20260811 02:51:22.024595 26720 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:22.025166 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 02:51:22.027256 26741 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:51:22.027405 26741 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:51:22.027683 26733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:22.028604 26733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:22.034636 26733 catalog_manager.cc:1347] Generated new cluster ID: a464d458bb76490cbe592ee432a98673
I20260811 02:51:22.034698 26733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:22.036315 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:22.027406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:49519 (local address 127.0.0.1:37299)
0811 02:51:22.027505 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:22.027509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:22.027850 (+ 341us) server_negotiation.cc:407] Connection header received
0811 02:51:22.027987 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:22.027990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:22.028035 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:22.028130 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:22.028817 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:22.029371 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:22.030187 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:22.030430 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:22.030553 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:22.032361 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:22.032367 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:22.032370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:22.032405 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:22.034215 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:22.034853 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:22.034856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:22.034858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:22.034951 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:22.035369 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:22.035372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:22.035383 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:22.035604 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:22.035755 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:22.036055 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 02:51:22.036159 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 02:51:22.059019 26733 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:22.060388 26733 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:22.068356 26733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc9144ef2a6c4f5b9cb72175d6add974: Generated new TSK 0
I20260811 02:51:22.069111 26733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:22.072880 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 26589
W20260811 02:51:22.084487 26563 connection.cc:535] client connection to 127.0.0.1:37299 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37299): Connection reset by peer (error 104)
2026-08-11T02:51:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1689 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:22 dist-test-slave-n326 krb5kdc[26759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:22 dist-test-slave-n326 krb5kdc[26759](info): set up 2 sockets
Aug 11 02:51:22 dist-test-slave-n326 krb5kdc[26759](info): commencing operation
krb5kdc: starting...
W20260811 02:51:23.379287 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:23 dist-test-slave-n326 krb5kdc[26759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:51:23.516270 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42527
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:23.627871 26781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:23.628180 26781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:23.628259 26781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:23.632237 26781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:23.632325 26781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:23.632365 26781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:23.632416 26781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:23.632452 26781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:23.637751 26781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42527
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:23.638863 26781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:23.639955 26781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:23.648722 26792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:23.648811 26781 server_base.cc:1034] running on GCE node
W20260811 02:51:23.648706 26789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:23.648706 26788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:23.649667 26781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:23.650763 26781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:23.651964 26781 hybrid_clock.cc:648] HybridClock initialized: now 1786416683651948 us; error 27 us; skew 500 ppm
Aug 11 02:51:23 dist-test-slave-n326 krb5kdc[26759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416683, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:23.654909 26781 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:51:23.655820 26781 webserver.cc:460] Webserver started at http://127.0.0.1:41105/ using document root <none> and password file <none>
I20260811 02:51:23.656342 26781 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:23.656412 26781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:23.656574 26781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:23.658538 26781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "9a99796ae33b4f3cb14a658776274fd7"
format_stamp: "Formatted at 2026-08-11 02:51:23 on dist-test-slave-n326"
server_key: "7ffdd80cc32fcc119d2652aa80d1aef6"
server_key_iv: "6d5c4f7afe0455b9affc5e3d2a376685"
server_key_version: "encryptionkey@0"
I20260811 02:51:23.659041 26781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "9a99796ae33b4f3cb14a658776274fd7"
format_stamp: "Formatted at 2026-08-11 02:51:23 on dist-test-slave-n326"
server_key: "7ffdd80cc32fcc119d2652aa80d1aef6"
server_key_iv: "6d5c4f7afe0455b9affc5e3d2a376685"
server_key_version: "encryptionkey@0"
I20260811 02:51:23.662703 26781 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:23.665235 26805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:23.666559 26781 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 02:51:23.666676 26781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "9a99796ae33b4f3cb14a658776274fd7"
format_stamp: "Formatted at 2026-08-11 02:51:23 on dist-test-slave-n326"
server_key: "7ffdd80cc32fcc119d2652aa80d1aef6"
server_key_iv: "6d5c4f7afe0455b9affc5e3d2a376685"
server_key_version: "encryptionkey@0"
I20260811 02:51:23.666771 26781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:23.682238 26781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:23.685161 26781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:23.685323 26781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:23.692718 26781 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42527
I20260811 02:51:23.692741 26893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42527 every 8 connection(s)
I20260811 02:51:23.693703 26781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:23.696506 26896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:23.702237 26896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7: Bootstrap starting.
I20260811 02:51:23.703037 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 26781
I20260811 02:51:23.703195 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:23.703536 22450 external_mini_cluster.cc:1428] Setting key 55d7f226e905e63bb70c7880aafb84dc
I20260811 02:51:23.705300 26896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:23.706015 26896 log.cc:824] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:23.708043 26896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7: No bootstrap required, opened a new log
I20260811 02:51:23.712517 26896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a99796ae33b4f3cb14a658776274fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42527 } }
I20260811 02:51:23.712791 26896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:23.712841 26896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a99796ae33b4f3cb14a658776274fd7, State: Initialized, Role: FOLLOWER
I20260811 02:51:23.712944 26902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:23.704784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:53207 (local address 127.0.0.1:42527)
0811 02:51:23.705202 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:23.705216 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:23.705250 (+ 34us) server_negotiation.cc:407] Connection header received
0811 02:51:23.705916 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:23.705931 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:23.706305 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:23.706677 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:23.707035 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:23.708028 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:23.709213 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:23.709679 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:23.709796 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:23.710797 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:23.710805 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:23.710826 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:23.711438 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:23.711507 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:23.711536 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:23.711674 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:23.711852 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 02:51:23.711986 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":131,"threads_started":1}
I20260811 02:51:23.713318 26896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a99796ae33b4f3cb14a658776274fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42527 } }
I20260811 02:51:23.713445 26896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:23.713521 26896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:23.713644 26896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:23.714504 26896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a99796ae33b4f3cb14a658776274fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42527 } }
I20260811 02:51:23.714879 26896 leader_election.cc:302] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a99796ae33b4f3cb14a658776274fd7; no voters:
I20260811 02:51:23.715260 26896 leader_election.cc:288] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:23.715324 26906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:23.715591 26906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [term 1 LEADER]: Becoming Leader. State: Replica: 9a99796ae33b4f3cb14a658776274fd7, State: Running, Role: LEADER
I20260811 02:51:23.715989 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:23.715962 26906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a99796ae33b4f3cb14a658776274fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42527 } }
I20260811 02:51:23.716665 26896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:23.717463 26907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a99796ae33b4f3cb14a658776274fd7. Latest consensus state: current_term: 1 leader_uuid: "9a99796ae33b4f3cb14a658776274fd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a99796ae33b4f3cb14a658776274fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42527 } } }
I20260811 02:51:23.717584 26907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:23.717819 26907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a99796ae33b4f3cb14a658776274fd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a99796ae33b4f3cb14a658776274fd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42527 } } }
I20260811 02:51:23.717909 26907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:23.718261 26929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:23.721568 26929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:23.724982 26902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:23.719515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:33201 (local address 127.0.0.1:42527)
0811 02:51:23.719624 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:23.719628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:23.719860 (+ 232us) server_negotiation.cc:407] Connection header received
0811 02:51:23.719972 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:23.719975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:23.720022 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:23.720114 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:23.720423 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:23.721228 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:23.722991 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:23.723402 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:23.723515 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:23.724466 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:23.724470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:23.724473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:23.724488 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:23.724520 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:23.724534 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:23.724625 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:23.724810 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 02:51:23.724847 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 02:51:23.727530 26929 catalog_manager.cc:1347] Generated new cluster ID: 194032db87bb46849969fc6090c5ea41
I20260811 02:51:23.727603 26929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:23.761200 26929 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:23.762512 26929 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:23.780957 26929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a99796ae33b4f3cb14a658776274fd7: Generated new TSK 0
I20260811 02:51:23.781853 26929 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:23.794492 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 26781
W20260811 02:51:23.817564 26755 connection.cc:535] client connection to 127.0.0.1:42527 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42527): Connection reset by peer (error 104)
2026-08-11T02:51:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1732 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:23 dist-test-slave-n326 krb5kdc[26946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:23 dist-test-slave-n326 krb5kdc[26946](info): set up 2 sockets
Aug 11 02:51:23 dist-test-slave-n326 krb5kdc[26946](info): commencing operation
krb5kdc: starting...
W20260811 02:51:25.143545 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:25 dist-test-slave-n326 krb5kdc[26946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416685, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 02:51:25.278632 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44001
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:25.392377 26962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:25.392661 26962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:25.392705 26962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:25.396631 26962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:25.396687 26962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:25.396706 26962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:25.396723 26962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:25.396737 26962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:25.401834 26962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44001
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:25.402868 26962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:25.403944 26962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:51:25.412654 26962 server_base.cc:1034] running on GCE node
W20260811 02:51:25.412591 26978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:25.412593 26972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:25.412595 26969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:25.413496 26962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:25.414685 26962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:25.415908 26962 hybrid_clock.cc:648] HybridClock initialized: now 1786416685415876 us; error 46 us; skew 500 ppm
Aug 11 02:51:25 dist-test-slave-n326 krb5kdc[26946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416685, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:25.425071 26962 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 02:51:25.426064 26962 webserver.cc:460] Webserver started at http://127.0.0.1:33091/ using document root <none> and password file <none>
I20260811 02:51:25.426690 26962 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:25.426792 26962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:25.427047 26962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:25.429365 26962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "e0fe8b2ea3cc470b908818fa9b17b17e"
format_stamp: "Formatted at 2026-08-11 02:51:25 on dist-test-slave-n326"
server_key: "8def34ed6de90308712701f064f86fba"
server_key_iv: "5b089ddfb0a4aba4dd76bb996cdb4c75"
server_key_version: "encryptionkey@0"
I20260811 02:51:25.429993 26962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "e0fe8b2ea3cc470b908818fa9b17b17e"
format_stamp: "Formatted at 2026-08-11 02:51:25 on dist-test-slave-n326"
server_key: "8def34ed6de90308712701f064f86fba"
server_key_iv: "5b089ddfb0a4aba4dd76bb996cdb4c75"
server_key_version: "encryptionkey@0"
I20260811 02:51:25.434158 26962 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 02:51:25.437076 26992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:25.438321 26962 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 02:51:25.438519 26962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "e0fe8b2ea3cc470b908818fa9b17b17e"
format_stamp: "Formatted at 2026-08-11 02:51:25 on dist-test-slave-n326"
server_key: "8def34ed6de90308712701f064f86fba"
server_key_iv: "5b089ddfb0a4aba4dd76bb996cdb4c75"
server_key_version: "encryptionkey@0"
I20260811 02:51:25.438644 26962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:25.443699 26962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:25.446732 26962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:25.446939 26962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:25.454861 26962 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44001
I20260811 02:51:25.454870 27096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44001 every 8 connection(s)
I20260811 02:51:25.455943 26962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:25.458703 27098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:25.464128 27098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e: Bootstrap starting.
I20260811 02:51:25.465889 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 26962
I20260811 02:51:25.466082 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:25.466385 22450 external_mini_cluster.cc:1428] Setting key a7c51ec747c329225b0d2bda4ed24590
I20260811 02:51:25.466588 27098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:25.467506 27098 log.cc:824] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:25.469470 27098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e: No bootstrap required, opened a new log
I20260811 02:51:25.471639 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:25.467552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:40457 (local address 127.0.0.1:44001)
0811 02:51:25.468033 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:25.468055 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:25.468094 (+ 39us) server_negotiation.cc:407] Connection header received
0811 02:51:25.468788 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:25.468817 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:25.469108 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:25.469406 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:25.470272 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:25.470277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:25.470288 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:25.470298 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:25.470347 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:25.470367 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:25.470476 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:25.470598 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 02:51:25.470785 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":157,"threads_started":1}
I20260811 02:51:25.473810 27098 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fe8b2ea3cc470b908818fa9b17b17e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44001 } }
I20260811 02:51:25.474107 27098 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:25.474205 27098 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0fe8b2ea3cc470b908818fa9b17b17e, State: Initialized, Role: FOLLOWER
I20260811 02:51:25.474825 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:25.474751 27098 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fe8b2ea3cc470b908818fa9b17b17e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44001 } }
I20260811 02:51:25.474938 27098 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:25.475009 27098 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:25.475133 27098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:25.476477 27098 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fe8b2ea3cc470b908818fa9b17b17e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44001 } }
I20260811 02:51:25.476883 27098 leader_election.cc:302] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0fe8b2ea3cc470b908818fa9b17b17e; no voters:
I20260811 02:51:25.477306 27098 leader_election.cc:288] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:25.477448 27119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:25.477720 27119 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [term 1 LEADER]: Becoming Leader. State: Replica: e0fe8b2ea3cc470b908818fa9b17b17e, State: Running, Role: LEADER
I20260811 02:51:25.478096 27119 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fe8b2ea3cc470b908818fa9b17b17e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44001 } }
I20260811 02:51:25.478770 27098 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:25.479904 27122 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0fe8b2ea3cc470b908818fa9b17b17e. Latest consensus state: current_term: 1 leader_uuid: "e0fe8b2ea3cc470b908818fa9b17b17e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fe8b2ea3cc470b908818fa9b17b17e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44001 } } }
I20260811 02:51:25.480053 27122 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:25.480432 27121 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0fe8b2ea3cc470b908818fa9b17b17e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0fe8b2ea3cc470b908818fa9b17b17e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44001 } } }
I20260811 02:51:25.480543 27121 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:25.481096 27109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:25.477412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.18:60225 (local address 127.0.0.1:44001)
0811 02:51:25.477533 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:25.477537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:25.479342 (+ 1805us) server_negotiation.cc:407] Connection header received
0811 02:51:25.479581 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:25.479584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:25.479622 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:25.479674 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:25.480586 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:25.480592 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:25.480597 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:25.480609 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:25.480636 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:25.480649 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:25.480707 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:25.480881 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 02:51:25.480931 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 02:51:25.484218 27146 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:51:25.484313 27146 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:51:25.484395 27133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:25.485100 27133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:25.490665 27133 catalog_manager.cc:1347] Generated new cluster ID: 7d118931091f4a4db57a3a4a9e43148a
I20260811 02:51:25.490726 27133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:25.499284 27133 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:25.499603 27133 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:25.508272 27133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0fe8b2ea3cc470b908818fa9b17b17e: Generated new TSK 0
I20260811 02:51:25.509095 27133 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:25.514231 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 26962
2026-08-11T02:51:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1714 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:25 dist-test-slave-n326 krb5kdc[27159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:25 dist-test-slave-n326 krb5kdc[27159](info): set up 2 sockets
Aug 11 02:51:25 dist-test-slave-n326 krb5kdc[27159](info): commencing operation
krb5kdc: starting...
W20260811 02:51:26.850977 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:26 dist-test-slave-n326 krb5kdc[27159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:51:26.982271 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:36253
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:44747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:36253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:27.094699 27186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:27.094976 27186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:27.095021 27186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:27.099010 27186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:27.099090 27186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:27.099112 27186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:27.099128 27186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:27.099143 27186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:27.104362 27186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:44747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:36253
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:36253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:27.105376 27186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:27.106354 27186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:27.114979 27198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:27.114991 27194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:27.115217 27186 server_base.cc:1034] running on GCE node
W20260811 02:51:27.114988 27195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:27.115883 27186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:27.116945 27186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:27.118134 27186 hybrid_clock.cc:648] HybridClock initialized: now 1786416687118118 us; error 32 us; skew 500 ppm
Aug 11 02:51:27 dist-test-slave-n326 krb5kdc[27159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416687, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:27.120918 27186 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:51:27.121827 27186 webserver.cc:460] Webserver started at http://127.21.236.190:34073/ using document root <none> and password file <none>
I20260811 02:51:27.122391 27186 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:27.122479 27186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:27.122716 27186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:27.124882 27186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "efcdee3a6beb42a682a3b3f752d31aea"
format_stamp: "Formatted at 2026-08-11 02:51:27 on dist-test-slave-n326"
server_key: "0b8c2997940006678d511bd473660ab4"
server_key_iv: "4b27ba939772eb951dc6e6559c09ce0e"
server_key_version: "encryptionkey@0"
I20260811 02:51:27.125491 27186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "efcdee3a6beb42a682a3b3f752d31aea"
format_stamp: "Formatted at 2026-08-11 02:51:27 on dist-test-slave-n326"
server_key: "0b8c2997940006678d511bd473660ab4"
server_key_iv: "4b27ba939772eb951dc6e6559c09ce0e"
server_key_version: "encryptionkey@0"
I20260811 02:51:27.129199 27186 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 02:51:27.131712 27216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:27.132966 27186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 02:51:27.133111 27186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "efcdee3a6beb42a682a3b3f752d31aea"
format_stamp: "Formatted at 2026-08-11 02:51:27 on dist-test-slave-n326"
server_key: "0b8c2997940006678d511bd473660ab4"
server_key_iv: "4b27ba939772eb951dc6e6559c09ce0e"
server_key_version: "encryptionkey@0"
I20260811 02:51:27.133224 27186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:27.149181 27186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:27.152194 27186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:27.152402 27186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:27.159657 27186 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:36253
I20260811 02:51:27.159709 27303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:36253 every 8 connection(s)
I20260811 02:51:27.160650 27186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:27.163532 27306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:27.168936 27306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea: Bootstrap starting.
I20260811 02:51:27.169566 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 27186
I20260811 02:51:27.169721 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:27.169979 22450 external_mini_cluster.cc:1428] Setting key 21a603bdbe2a2c4da77b31fe594c209e
I20260811 02:51:27.171749 27306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:27.172495 27306 log.cc:824] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:27.174440 27306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea: No bootstrap required, opened a new log
I20260811 02:51:27.177387 27306 raft_consensus.cc:348] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efcdee3a6beb42a682a3b3f752d31aea" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36253 } }
I20260811 02:51:27.177641 27306 raft_consensus.cc:374] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:27.177731 27306 raft_consensus.cc:729] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efcdee3a6beb42a682a3b3f752d31aea, State: Initialized, Role: FOLLOWER
I20260811 02:51:27.178201 27306 consensus_queue.cc:260] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efcdee3a6beb42a682a3b3f752d31aea" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36253 } }
I20260811 02:51:27.178356 27306 raft_consensus.cc:388] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:27.178448 27306 raft_consensus.cc:482] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:27.178548 27306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [term 0 FOLLOWER]: Advancing to term 1
Aug 11 02:51:27 dist-test-slave-n326 krb5kdc[27159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:51:27.179703 27306 raft_consensus.cc:504] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efcdee3a6beb42a682a3b3f752d31aea" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36253 } }
I20260811 02:51:27.180055 27306 leader_election.cc:302] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efcdee3a6beb42a682a3b3f752d31aea; no voters:
I20260811 02:51:27.180368 27306 leader_election.cc:288] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:27.180563 27317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:27.180871 27317 raft_consensus.cc:686] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [term 1 LEADER]: Becoming Leader. State: Replica: efcdee3a6beb42a682a3b3f752d31aea, State: Running, Role: LEADER
I20260811 02:51:27.181254 27317 consensus_queue.cc:237] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efcdee3a6beb42a682a3b3f752d31aea" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36253 } }
I20260811 02:51:27.181864 27306 sys_catalog.cc:564] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:27.183238 27320 sys_catalog.cc:455] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [sys.catalog]: SysCatalogTable state changed. Reason: New leader efcdee3a6beb42a682a3b3f752d31aea. Latest consensus state: current_term: 1 leader_uuid: "efcdee3a6beb42a682a3b3f752d31aea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efcdee3a6beb42a682a3b3f752d31aea" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36253 } } }
I20260811 02:51:27.183450 27320 sys_catalog.cc:458] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:27.183948 27337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:27.184900 27319 sys_catalog.cc:455] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efcdee3a6beb42a682a3b3f752d31aea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efcdee3a6beb42a682a3b3f752d31aea" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36253 } } }
I20260811 02:51:27.185040 27319 sys_catalog.cc:458] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:27.187290 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:27.171220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48173 (local address 127.21.236.190:36253)
0811 02:51:27.171890 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:27.171907 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:27.171945 (+ 38us) server_negotiation.cc:407] Connection header received
0811 02:51:27.172674 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:27.172692 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:27.173065 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:27.173393 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:27.174244 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:27.175331 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:27.176216 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:27.177215 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:27.177395 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:27.180649 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:27.180659 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:27.180681 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:27.180722 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:27.183527 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:27.184291 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:27.184297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:27.184304 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:27.184454 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:27.184892 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:27.184896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:27.184898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:27.185487 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:27.185719 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:27.186049 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 02:51:27.186286 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":126,"threads_started":1}
I20260811 02:51:27.187981 27337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:27.190770 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:27.194967 27337 catalog_manager.cc:1347] Generated new cluster ID: bfe55a98f4554cb58fff9dee881f13ea
I20260811 02:51:27.195032 27337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:27.202354 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:27.193104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54007 (local address 127.21.236.190:36253)
0811 02:51:27.193210 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:27.193214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:27.193535 (+ 321us) server_negotiation.cc:407] Connection header received
0811 02:51:27.193660 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:27.193665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:27.193718 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:27.193807 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:27.194656 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:27.195528 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:27.196669 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:27.197055 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:27.197167 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:27.198744 (+ 1577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:27.198748 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:27.198751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:27.198777 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:27.200525 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:27.201051 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:27.201054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:27.201056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:27.201144 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:27.201473 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:27.201477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:27.201479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:27.201689 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:27.201829 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:27.202093 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 02:51:27.202193 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 02:51:27.210441 27337 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:27.211498 27337 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:27.217096 27337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efcdee3a6beb42a682a3b3f752d31aea: Generated new TSK 0
I20260811 02:51:27.217885 27337 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:27.222697 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 27186
W20260811 02:51:27.238090 27153 connection.cc:535] client connection to 127.21.236.190:36253 recv error: Network error: failed to read from TLS socket (remote: 127.21.236.190:36253): Connection reset by peer (error 104)
2026-08-11T02:51:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1705 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:27 dist-test-slave-n326 krb5kdc[27367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:27 dist-test-slave-n326 krb5kdc[27367](info): set up 2 sockets
Aug 11 02:51:27 dist-test-slave-n326 krb5kdc[27367](info): commencing operation
krb5kdc: starting...
W20260811 02:51:28.545212 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:28 dist-test-slave-n326 krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:51:28.681887 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:43207
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:41971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:43207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:28.803853 27390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:28.804268 27390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:28.804370 27390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:28.808521 27390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:28.808585 27390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:28.808606 27390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:28.808622 27390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:28.808638 27390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:28.813810 27390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:41971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:43207
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:43207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:28.814769 27390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:28.815783 27390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 02:51:28.824635 27390 server_base.cc:1034] running on GCE node
W20260811 02:51:28.824592 27401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:28.824580 27403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:28.824559 27406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:28.825394 27390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:28.826546 27390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:28.827754 27390 hybrid_clock.cc:648] HybridClock initialized: now 1786416688827718 us; error 46 us; skew 500 ppm
Aug 11 02:51:28 dist-test-slave-n326 krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416688, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:28.830816 27390 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:51:28.831888 27390 webserver.cc:460] Webserver started at http://127.21.236.190:33893/ using document root <none> and password file <none>
I20260811 02:51:28.832450 27390 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:28.832525 27390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:28.832706 27390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:28.834928 27390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "dfd0a92d7d364c22afb11e4a2084f045"
format_stamp: "Formatted at 2026-08-11 02:51:28 on dist-test-slave-n326"
server_key: "e56fade6973e8755f8f81f2be3e7ae37"
server_key_iv: "a298d555037889242572a830527d6b82"
server_key_version: "encryptionkey@0"
I20260811 02:51:28.835546 27390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "dfd0a92d7d364c22afb11e4a2084f045"
format_stamp: "Formatted at 2026-08-11 02:51:28 on dist-test-slave-n326"
server_key: "e56fade6973e8755f8f81f2be3e7ae37"
server_key_iv: "a298d555037889242572a830527d6b82"
server_key_version: "encryptionkey@0"
I20260811 02:51:28.839308 27390 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 02:51:28.841778 27419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:28.843206 27390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:28.843359 27390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "dfd0a92d7d364c22afb11e4a2084f045"
format_stamp: "Formatted at 2026-08-11 02:51:28 on dist-test-slave-n326"
server_key: "e56fade6973e8755f8f81f2be3e7ae37"
server_key_iv: "a298d555037889242572a830527d6b82"
server_key_version: "encryptionkey@0"
I20260811 02:51:28.843477 27390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:28.872313 27390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:28.875761 27390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:28.875974 27390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:28.883582 27390 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:43207
I20260811 02:51:28.883610 27507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:43207 every 8 connection(s)
I20260811 02:51:28.884545 27390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:28.887501 27508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:28.889462 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 27390
I20260811 02:51:28.889568 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:28.889850 22450 external_mini_cluster.cc:1428] Setting key cf4587ccbd14ad7fd2d23501c9cd841d
I20260811 02:51:28.894778 27508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045: Bootstrap starting.
I20260811 02:51:28.897485 27508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:28.898326 27508 log.cc:824] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:28.900429 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:28.891316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60543 (local address 127.21.236.190:43207)
0811 02:51:28.891763 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:28.891785 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:28.891831 (+ 46us) server_negotiation.cc:407] Connection header received
0811 02:51:28.892770 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:28.892794 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:28.893248 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:28.893628 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:28.894091 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:28.895676 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:28.896547 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:28.897271 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:28.897460 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:28.898274 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:28.898281 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:28.898296 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:28.898821 (+ 525us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:28.898887 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:28.898919 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:28.899030 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:28.899339 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 02:51:28.899472 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":121,"threads_started":1}
I20260811 02:51:28.900699 27508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045: No bootstrap required, opened a new log
I20260811 02:51:28.903278 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:28.904889 27508 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfd0a92d7d364c22afb11e4a2084f045" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43207 } }
I20260811 02:51:28.905274 27508 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:28.905417 27508 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfd0a92d7d364c22afb11e4a2084f045, State: Initialized, Role: FOLLOWER
I20260811 02:51:28.905889 27508 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfd0a92d7d364c22afb11e4a2084f045" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43207 } }
I20260811 02:51:28.906051 27508 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:28.906129 27508 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:28.906256 27508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:28.907223 27508 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfd0a92d7d364c22afb11e4a2084f045" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43207 } }
I20260811 02:51:28.907564 27508 leader_election.cc:302] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfd0a92d7d364c22afb11e4a2084f045; no voters:
I20260811 02:51:28.907892 27508 leader_election.cc:288] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:28.907984 27523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:28.908224 27523 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [term 1 LEADER]: Becoming Leader. State: Replica: dfd0a92d7d364c22afb11e4a2084f045, State: Running, Role: LEADER
I20260811 02:51:28.908565 27523 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfd0a92d7d364c22afb11e4a2084f045" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43207 } }
I20260811 02:51:28.909227 27508 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:28.909446 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:28.905274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33343 (local address 127.21.236.190:43207)
0811 02:51:28.905377 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:28.905381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:28.905590 (+ 209us) server_negotiation.cc:407] Connection header received
0811 02:51:28.905705 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:28.905710 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:28.905751 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:28.905831 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:28.906099 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:28.906708 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:28.907621 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:28.907972 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:28.908080 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:28.908983 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:28.908986 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:28.908989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:28.909001 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:28.909029 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:28.909039 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:28.909132 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:28.909287 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 02:51:28.909320 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 02:51:28.909921 27523 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfd0a92d7d364c22afb11e4a2084f045. Latest consensus state: current_term: 1 leader_uuid: "dfd0a92d7d364c22afb11e4a2084f045" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfd0a92d7d364c22afb11e4a2084f045" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43207 } } }
I20260811 02:51:28.910070 27523 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:28.910867 27527 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfd0a92d7d364c22afb11e4a2084f045" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfd0a92d7d364c22afb11e4a2084f045" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 43207 } } }
I20260811 02:51:28.911099 27527 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:28.911029 27533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:28.914232 27533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:28.919624 27533 catalog_manager.cc:1347] Generated new cluster ID: 5a81dc2704614b25bc204b0470c65a4c
I20260811 02:51:28.919696 27533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:28.936399 27533 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:28.937700 27533 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:28.950717 27533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfd0a92d7d364c22afb11e4a2084f045: Generated new TSK 0
I20260811 02:51:28.951495 27533 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:28.963191 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 27390
W20260811 02:51:28.976143 27364 connection.cc:535] client connection to 127.21.236.190:43207 recv error: Network error: failed to read from TLS socket (remote: 127.21.236.190:43207): Connection reset by peer (error 104)
2026-08-11T02:51:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1738 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:29 dist-test-slave-n326 krb5kdc[27562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:29 dist-test-slave-n326 krb5kdc[27562](info): set up 2 sockets
Aug 11 02:51:29 dist-test-slave-n326 krb5kdc[27562](info): commencing operation
krb5kdc: starting...
W20260811 02:51:30.306622 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:30 dist-test-slave-n326 krb5kdc[27562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:51:30.439020 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:37885
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:40879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:37885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:30.552184 27584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:30.552505 27584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:30.552582 27584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:30.556639 27584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:30.556723 27584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:30.556761 27584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:30.556815 27584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:30.556855 27584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:30.562107 27584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:40879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:37885
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:37885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:30.563242 27584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:30.564286 27584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:30.573312 27596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:30.573338 27598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:30.573386 27592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:30.573530 27584 server_base.cc:1034] running on GCE node
I20260811 02:51:30.574226 27584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:30.575330 27584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:30.576501 27584 hybrid_clock.cc:648] HybridClock initialized: now 1786416690576472 us; error 47 us; skew 500 ppm
Aug 11 02:51:30 dist-test-slave-n326 krb5kdc[27562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416690, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:30.582026 27584 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:51:30.582970 27584 webserver.cc:460] Webserver started at http://127.21.236.190:42667/ using document root <none> and password file <none>
I20260811 02:51:30.583590 27584 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:30.583659 27584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:30.583835 27584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:30.585939 27584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "1e9ee7472a8a4257ad300abaa0985eb8"
format_stamp: "Formatted at 2026-08-11 02:51:30 on dist-test-slave-n326"
server_key: "2eea0aabf12588e973d54f600558da12"
server_key_iv: "6ca9bc85a58303aa9daf63c27818867a"
server_key_version: "encryptionkey@0"
I20260811 02:51:30.586524 27584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "1e9ee7472a8a4257ad300abaa0985eb8"
format_stamp: "Formatted at 2026-08-11 02:51:30 on dist-test-slave-n326"
server_key: "2eea0aabf12588e973d54f600558da12"
server_key_iv: "6ca9bc85a58303aa9daf63c27818867a"
server_key_version: "encryptionkey@0"
I20260811 02:51:30.590639 27584 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:51:30.593641 27610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:30.595191 27584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:30.595332 27584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "1e9ee7472a8a4257ad300abaa0985eb8"
format_stamp: "Formatted at 2026-08-11 02:51:30 on dist-test-slave-n326"
server_key: "2eea0aabf12588e973d54f600558da12"
server_key_iv: "6ca9bc85a58303aa9daf63c27818867a"
server_key_version: "encryptionkey@0"
I20260811 02:51:30.595458 27584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:30.600266 27584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:30.603456 27584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:30.603668 27584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:30.611129 27584 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:37885
I20260811 02:51:30.611163 27708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:37885 every 8 connection(s)
I20260811 02:51:30.612092 27584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:30.614881 27709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:30.616360 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 27584
I20260811 02:51:30.616503 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:30.616823 22450 external_mini_cluster.cc:1428] Setting key 04c02081db0fa2c359ff654a2f72f038
I20260811 02:51:30.621551 27709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8: Bootstrap starting.
I20260811 02:51:30.622123 27715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:30.618240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58101 (local address 127.21.236.190:37885)
0811 02:51:30.618700 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:30.618717 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:30.618752 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:51:30.619422 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:30.619441 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:30.619727 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:30.619999 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:30.620843 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:30.620848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:30.620859 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:30.620870 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:30.620924 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:30.620943 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:30.621054 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:30.621163 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 02:51:30.621330 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":145,"threads_started":1}
I20260811 02:51:30.623822 27709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:30.624579 27709 log.cc:824] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:30.624778 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:30.626856 27709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8: No bootstrap required, opened a new log
I20260811 02:51:30.628574 27715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:30.626964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60961 (local address 127.21.236.190:37885)
0811 02:51:30.627136 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:30.627141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:30.627320 (+ 179us) server_negotiation.cc:407] Connection header received
0811 02:51:30.627415 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:30.627418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:30.627459 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:30.627536 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:30.628217 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:30.628221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:30.628224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:30.628235 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:30.628263 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:30.628275 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:30.628371 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:30.628417 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 02:51:30.628458 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:51:30.630774 27709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9ee7472a8a4257ad300abaa0985eb8" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 37885 } }
I20260811 02:51:30.630995 27709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:30.631031 27709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e9ee7472a8a4257ad300abaa0985eb8, State: Initialized, Role: FOLLOWER
I20260811 02:51:30.631438 27709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9ee7472a8a4257ad300abaa0985eb8" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 37885 } }
I20260811 02:51:30.631556 27709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:30.631603 27709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:30.631711 27709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:30.632553 27709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9ee7472a8a4257ad300abaa0985eb8" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 37885 } }
I20260811 02:51:30.632884 27709 leader_election.cc:302] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e9ee7472a8a4257ad300abaa0985eb8; no voters:
I20260811 02:51:30.633255 27709 leader_election.cc:288] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:30.633327 27725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:30.633572 27725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [term 1 LEADER]: Becoming Leader. State: Replica: 1e9ee7472a8a4257ad300abaa0985eb8, State: Running, Role: LEADER
I20260811 02:51:30.633891 27725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9ee7472a8a4257ad300abaa0985eb8" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 37885 } }
I20260811 02:51:30.634596 27709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:30.635710 27726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e9ee7472a8a4257ad300abaa0985eb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9ee7472a8a4257ad300abaa0985eb8" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 37885 } } }
I20260811 02:51:30.635747 27727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e9ee7472a8a4257ad300abaa0985eb8. Latest consensus state: current_term: 1 leader_uuid: "1e9ee7472a8a4257ad300abaa0985eb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e9ee7472a8a4257ad300abaa0985eb8" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 37885 } } }
I20260811 02:51:30.635924 27727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:30.635914 27726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:30.636344 27745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:30.639423 27745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:30.645123 27745 catalog_manager.cc:1347] Generated new cluster ID: c3111c6fb00247a3b741821d5f1feb8a
I20260811 02:51:30.645234 27745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:30.658726 27745 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:30.659123 27745 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:30.664917 27745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e9ee7472a8a4257ad300abaa0985eb8: Generated new TSK 0
I20260811 02:51:30.665719 27745 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:30.670711 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 27584
2026-08-11T02:51:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1705 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:30 dist-test-slave-n326 krb5kdc[27771](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:30 dist-test-slave-n326 krb5kdc[27771](info): set up 2 sockets
Aug 11 02:51:30 dist-test-slave-n326 krb5kdc[27771](info): commencing operation
krb5kdc: starting...
W20260811 02:51:32.048846 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.352s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:32 dist-test-slave-n326 krb5kdc[27771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:51:32.181700 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:46139
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:45329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:46139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:32.300249 27790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:32.300519 27790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:32.300562 27790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:32.304471 27790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:32.304525 27790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:32.304544 27790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:32.304561 27790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:32.304576 27790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:32.309759 27790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:45329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:46139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:46139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:32.310734 27790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:32.311697 27790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:32.320096 27802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:32.320096 27809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:32.321861 27805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:32.322268 27790 server_base.cc:1034] running on GCE node
I20260811 02:51:32.322796 27790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:32.323910 27790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:32.325079 27790 hybrid_clock.cc:648] HybridClock initialized: now 1786416692325055 us; error 38 us; skew 500 ppm
Aug 11 02:51:32 dist-test-slave-n326 krb5kdc[27771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416692, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:32.328040 27790 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:51:32.329012 27790 webserver.cc:460] Webserver started at http://127.21.236.190:46371/ using document root <none> and password file <none>
I20260811 02:51:32.329682 27790 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:32.329761 27790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:32.329995 27790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:32.332331 27790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "5f4577e49cd24b7f836d1263c77af275"
format_stamp: "Formatted at 2026-08-11 02:51:32 on dist-test-slave-n326"
server_key: "031679ebcdb6aac798b8b3627d8c40be"
server_key_iv: "611d6d3b1dd3a0367dd0d40fd6edbe29"
server_key_version: "encryptionkey@0"
I20260811 02:51:32.332955 27790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "5f4577e49cd24b7f836d1263c77af275"
format_stamp: "Formatted at 2026-08-11 02:51:32 on dist-test-slave-n326"
server_key: "031679ebcdb6aac798b8b3627d8c40be"
server_key_iv: "611d6d3b1dd3a0367dd0d40fd6edbe29"
server_key_version: "encryptionkey@0"
I20260811 02:51:32.336992 27790 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 02:51:32.339655 27819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:32.341009 27790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:32.341176 27790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "5f4577e49cd24b7f836d1263c77af275"
format_stamp: "Formatted at 2026-08-11 02:51:32 on dist-test-slave-n326"
server_key: "031679ebcdb6aac798b8b3627d8c40be"
server_key_iv: "611d6d3b1dd3a0367dd0d40fd6edbe29"
server_key_version: "encryptionkey@0"
I20260811 02:51:32.341290 27790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:32.353960 27790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:32.357183 27790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:32.357383 27790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:32.365147 27790 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:46139
I20260811 02:51:32.365259 27905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:46139 every 8 connection(s)
I20260811 02:51:32.366130 27790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:32.368577 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 27790
I20260811 02:51:32.368742 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:32.369084 22450 external_mini_cluster.cc:1428] Setting key 293c53c1e79c80edb292994857a66a94
I20260811 02:51:32.369375 27907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:32.376958 27907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275: Bootstrap starting.
Aug 11 02:51:32 dist-test-slave-n326 krb5kdc[27771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:51:32.379714 27907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:32.380524 27907 log.cc:824] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:32.382951 27907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275: No bootstrap required, opened a new log
I20260811 02:51:32.385515 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:32.370520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40707 (local address 127.21.236.190:46139)
0811 02:51:32.370925 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:32.370941 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:32.370976 (+ 35us) server_negotiation.cc:407] Connection header received
0811 02:51:32.371676 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:32.371693 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:32.372025 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:32.372296 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:32.373355 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:32.374353 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:32.375842 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:32.376372 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:32.376524 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:32.379865 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:32.379874 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:32.379909 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:32.379943 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:32.382121 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:32.382682 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:32.382688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:32.382693 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:32.382836 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:32.383200 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:32.383204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:32.383206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:32.383717 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:32.383913 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:32.384207 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 02:51:32.384500 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":141,"threads_started":1}
I20260811 02:51:32.387568 27907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4577e49cd24b7f836d1263c77af275" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46139 } }
I20260811 02:51:32.387902 27907 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:32.388085 27907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f4577e49cd24b7f836d1263c77af275, State: Initialized, Role: FOLLOWER
I20260811 02:51:32.388645 27907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4577e49cd24b7f836d1263c77af275" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46139 } }
I20260811 02:51:32.388790 27907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:32.388674 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:32.388832 27907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:32.388942 27907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:32.389882 27907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4577e49cd24b7f836d1263c77af275" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46139 } }
I20260811 02:51:32.390264 27907 leader_election.cc:302] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f4577e49cd24b7f836d1263c77af275; no voters:
I20260811 02:51:32.390627 27907 leader_election.cc:288] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:32.390712 27928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:32.390955 27928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [term 1 LEADER]: Becoming Leader. State: Replica: 5f4577e49cd24b7f836d1263c77af275, State: Running, Role: LEADER
I20260811 02:51:32.391338 27928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4577e49cd24b7f836d1263c77af275" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46139 } }
I20260811 02:51:32.392025 27907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:32.392537 27928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f4577e49cd24b7f836d1263c77af275. Latest consensus state: current_term: 1 leader_uuid: "5f4577e49cd24b7f836d1263c77af275" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4577e49cd24b7f836d1263c77af275" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46139 } } }
I20260811 02:51:32.393528 27928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:32.393374 27933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f4577e49cd24b7f836d1263c77af275" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f4577e49cd24b7f836d1263c77af275" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 46139 } } }
I20260811 02:51:32.394510 27933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:32.394865 27947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:32.398399 27947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:32.401130 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:32.390909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52367 (local address 127.21.236.190:46139)
0811 02:51:32.391032 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:32.391037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:32.391333 (+ 296us) server_negotiation.cc:407] Connection header received
0811 02:51:32.391547 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:32.391550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:32.391599 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:32.391654 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:32.392539 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:32.393389 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:32.394596 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:32.394956 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:32.395036 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:32.396973 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:32.396977 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:32.396980 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:32.397006 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:32.398871 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:32.399420 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:32.399424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:32.399426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:32.399646 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:32.400101 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:32.400105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:32.400106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:32.400363 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:32.400506 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:32.400818 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 02:51:32.400953 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:51:32.406193 27947 catalog_manager.cc:1347] Generated new cluster ID: 6c2146c319614fe3b631be9a4128c094
I20260811 02:51:32.406266 27947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:32.421792 27947 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:32.423193 27947 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:32.432242 27947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f4577e49cd24b7f836d1263c77af275: Generated new TSK 0
I20260811 02:51:32.433136 27947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:32.443552 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 27790
2026-08-11T02:51:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1781 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:32 dist-test-slave-n326 krb5kdc[27962](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:32 dist-test-slave-n326 krb5kdc[27962](info): set up 2 sockets
Aug 11 02:51:32 dist-test-slave-n326 krb5kdc[27962](info): commencing operation
krb5kdc: starting...
W20260811 02:51:33.804327 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:33 dist-test-slave-n326 krb5kdc[27962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:51:33.947801 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:38681
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:34799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:38681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:34.067744 27982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:34.068241 27982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:34.068329 27982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:34.072499 27982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:34.072572 27982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:34.072592 27982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:34.072619 27982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:34.072635 27982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:34.080478 27982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:34799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:38681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:38681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:34.082024 27982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:34.083714 27982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:34.093672 27990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:34.093708 27992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:34.093672 27995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:34.093750 27982 server_base.cc:1034] running on GCE node
I20260811 02:51:34.094450 27982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:34.095654 27982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:34.096846 27982 hybrid_clock.cc:648] HybridClock initialized: now 1786416694096820 us; error 38 us; skew 500 ppm
Aug 11 02:51:34 dist-test-slave-n326 krb5kdc[27962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416694, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:34.099767 27982 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:51:34.100754 27982 webserver.cc:460] Webserver started at http://127.21.236.190:41123/ using document root <none> and password file <none>
I20260811 02:51:34.101336 27982 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:34.101456 27982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:34.101707 27982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:34.103883 27982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "8e75750fbe704270b47b267991a10b37"
format_stamp: "Formatted at 2026-08-11 02:51:34 on dist-test-slave-n326"
server_key: "a84e8fce4f9a3d8cc1ec5ee494a09776"
server_key_iv: "6390254c10367061dcda7300342374d3"
server_key_version: "encryptionkey@0"
I20260811 02:51:34.104477 27982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "8e75750fbe704270b47b267991a10b37"
format_stamp: "Formatted at 2026-08-11 02:51:34 on dist-test-slave-n326"
server_key: "a84e8fce4f9a3d8cc1ec5ee494a09776"
server_key_iv: "6390254c10367061dcda7300342374d3"
server_key_version: "encryptionkey@0"
I20260811 02:51:34.108326 27982 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:51:34.111135 28005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:34.112699 27982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:51:34.112861 27982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "8e75750fbe704270b47b267991a10b37"
format_stamp: "Formatted at 2026-08-11 02:51:34 on dist-test-slave-n326"
server_key: "a84e8fce4f9a3d8cc1ec5ee494a09776"
server_key_iv: "6390254c10367061dcda7300342374d3"
server_key_version: "encryptionkey@0"
I20260811 02:51:34.112990 27982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:34.158301 27982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:34.162040 27982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:34.162286 27982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:34.169848 27982 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:38681
I20260811 02:51:34.170183 28115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:38681 every 8 connection(s)
I20260811 02:51:34.170902 27982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:34.173899 28116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:34.176862 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 27982
I20260811 02:51:34.177029 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:34.177350 22450 external_mini_cluster.cc:1428] Setting key 8264a5e465b017a6ebc674cebe8abd5c
I20260811 02:51:34.181102 28116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37: Bootstrap starting.
I20260811 02:51:34.183444 28116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:34.184221 28116 log.cc:824] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:34.186517 28116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37: No bootstrap required, opened a new log
I20260811 02:51:34.187697 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:34.178873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46519 (local address 127.21.236.190:38681)
0811 02:51:34.179390 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:34.179415 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:34.179481 (+ 66us) server_negotiation.cc:407] Connection header received
0811 02:51:34.180292 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:34.180313 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:34.180691 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:34.181084 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:34.181589 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:34.182652 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:34.183833 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:34.184418 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:34.184650 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:34.185628 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:34.185634 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:34.185648 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:34.186095 (+ 447us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:34.186166 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:34.186193 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:34.186330 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:34.186670 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 02:51:34.186898 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":161,"threads_started":1}
I20260811 02:51:34.191006 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:34.190905 28116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e75750fbe704270b47b267991a10b37" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38681 } }
I20260811 02:51:34.191231 28116 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:34.191324 28116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e75750fbe704270b47b267991a10b37, State: Initialized, Role: FOLLOWER
I20260811 02:51:34.191747 28116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e75750fbe704270b47b267991a10b37" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38681 } }
I20260811 02:51:34.191906 28116 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:34.191977 28116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:34.192082 28116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:34.193209 28116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e75750fbe704270b47b267991a10b37" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38681 } }
I20260811 02:51:34.193619 28116 leader_election.cc:302] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e75750fbe704270b47b267991a10b37; no voters:
I20260811 02:51:34.194000 28116 leader_election.cc:288] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:34.194087 28133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:34.194285 28133 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [term 1 LEADER]: Becoming Leader. State: Replica: 8e75750fbe704270b47b267991a10b37, State: Running, Role: LEADER
I20260811 02:51:34.194655 28133 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e75750fbe704270b47b267991a10b37" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38681 } }
I20260811 02:51:34.195396 28116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:34.196486 28134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e75750fbe704270b47b267991a10b37. Latest consensus state: current_term: 1 leader_uuid: "8e75750fbe704270b47b267991a10b37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e75750fbe704270b47b267991a10b37" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38681 } } }
I20260811 02:51:34.196623 28134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:34.197042 28138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e75750fbe704270b47b267991a10b37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e75750fbe704270b47b267991a10b37" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 38681 } } }
I20260811 02:51:34.197214 28138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:34.198223 28151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:34.198851 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:34.193499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38371 (local address 127.21.236.190:38681)
0811 02:51:34.193637 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:34.193642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:34.193811 (+ 169us) server_negotiation.cc:407] Connection header received
0811 02:51:34.194033 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:34.194037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:34.194127 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:34.194222 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:34.194693 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:34.195408 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:34.196696 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:34.197042 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:34.197125 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:34.198306 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:34.198312 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:34.198317 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:34.198333 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:34.198367 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:34.198381 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:34.198445 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:34.198654 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 02:51:34.198715 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 02:51:34.201985 28151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:34.208801 28151 catalog_manager.cc:1347] Generated new cluster ID: 12817757311b49b687666ece13aa6ecd
I20260811 02:51:34.208894 28151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:34.241603 28151 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:34.242946 28151 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:34.254882 28151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e75750fbe704270b47b267991a10b37: Generated new TSK 0
I20260811 02:51:34.255762 28151 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:34.269064 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 27982
2026-08-11T02:51:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1820 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:34 dist-test-slave-n326 krb5kdc[28168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:34 dist-test-slave-n326 krb5kdc[28168](info): set up 2 sockets
Aug 11 02:51:34 dist-test-slave-n326 krb5kdc[28168](info): commencing operation
krb5kdc: starting...
W20260811 02:51:35.602802 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:35 dist-test-slave-n326 krb5kdc[28168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:51:35.739863 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:36421
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:43533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:36421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:35.858011 28193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:35.858309 28193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:35.858352 28193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:35.862308 28193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:35.862371 28193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:35.862398 28193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:35.862416 28193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:35.862430 28193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:35.867795 28193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:43533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:36421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:36421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:35.868844 28193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:35.869843 28193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:35.879205 28202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:35.879249 28193 server_base.cc:1034] running on GCE node
W20260811 02:51:35.879166 28203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:35.879189 28206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:35.880121 28193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:35.881212 28193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:35.882376 28193 hybrid_clock.cc:648] HybridClock initialized: now 1786416695882372 us; error 47 us; skew 500 ppm
Aug 11 02:51:35 dist-test-slave-n326 krb5kdc[28168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416695, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:35.890152 28193 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:51:35.891390 28193 webserver.cc:460] Webserver started at http://127.21.236.190:43985/ using document root <none> and password file <none>
I20260811 02:51:35.892100 28193 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:35.892210 28193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:35.892479 28193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:35.894800 28193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "b2dfbb45d52e4cc7b03c41b8bb870272"
format_stamp: "Formatted at 2026-08-11 02:51:35 on dist-test-slave-n326"
server_key: "6e7445e5cb5142c37dc3b5cbdd064bed"
server_key_iv: "ff1901ee736eb485f847c69cf74140fe"
server_key_version: "encryptionkey@0"
I20260811 02:51:35.895503 28193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "b2dfbb45d52e4cc7b03c41b8bb870272"
format_stamp: "Formatted at 2026-08-11 02:51:35 on dist-test-slave-n326"
server_key: "6e7445e5cb5142c37dc3b5cbdd064bed"
server_key_iv: "ff1901ee736eb485f847c69cf74140fe"
server_key_version: "encryptionkey@0"
I20260811 02:51:35.899725 28193 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:51:35.902585 28220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:35.904269 28193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:35.904414 28193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "b2dfbb45d52e4cc7b03c41b8bb870272"
format_stamp: "Formatted at 2026-08-11 02:51:35 on dist-test-slave-n326"
server_key: "6e7445e5cb5142c37dc3b5cbdd064bed"
server_key_iv: "ff1901ee736eb485f847c69cf74140fe"
server_key_version: "encryptionkey@0"
I20260811 02:51:35.904605 28193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:35.909425 28193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:35.912750 28193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:35.912969 28193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:35.920569 28193 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:36421
I20260811 02:51:35.920598 28308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:36421 every 8 connection(s)
I20260811 02:51:35.921562 28193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:35.924692 28309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:35.927845 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 28193
I20260811 02:51:35.928004 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:35.928328 22450 external_mini_cluster.cc:1428] Setting key 445e6fcfe17b68e957e99fe1f72c61c7
I20260811 02:51:35.931777 28309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272: Bootstrap starting.
I20260811 02:51:35.934201 28313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:35.929971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51495 (local address 127.21.236.190:36421)
0811 02:51:35.930485 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:35.930500 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:35.930538 (+ 38us) server_negotiation.cc:407] Connection header received
0811 02:51:35.931275 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:35.931295 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:35.931603 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:35.931937 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:35.932892 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:35.932897 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:35.932909 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:35.932921 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:35.932984 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:35.933005 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:35.933126 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:35.933272 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 02:51:35.933458 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":140,"threads_started":1}
I20260811 02:51:35.934775 28309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:35.935671 28309 log.cc:824] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:35.937387 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:35.938015 28309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272: No bootstrap required, opened a new log
I20260811 02:51:35.941762 28313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:35.939812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48915 (local address 127.21.236.190:36421)
0811 02:51:35.939966 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:35.939971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:35.940207 (+ 236us) server_negotiation.cc:407] Connection header received
0811 02:51:35.940345 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:35.940347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:35.940380 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:35.940495 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:35.941275 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:35.941278 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:35.941280 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 02:51:35.941291 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:35.941317 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 02:51:35.941329 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:35.941457 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:35.941589 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 02:51:35.941623 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 02:51:35.943126 28309 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2dfbb45d52e4cc7b03c41b8bb870272" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36421 } }
I20260811 02:51:35.943416 28309 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 02:51:35.943493 28309 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2dfbb45d52e4cc7b03c41b8bb870272, State: Initialized, Role: FOLLOWER
I20260811 02:51:35.943928 28309 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2dfbb45d52e4cc7b03c41b8bb870272" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36421 } }
I20260811 02:51:35.944103 28309 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 02:51:35.944202 28309 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 02:51:35.944319 28309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:35.945365 28309 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2dfbb45d52e4cc7b03c41b8bb870272" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36421 } }
I20260811 02:51:35.945748 28309 leader_election.cc:302] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2dfbb45d52e4cc7b03c41b8bb870272; no voters:
I20260811 02:51:35.946117 28309 leader_election.cc:288] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 02:51:35.946283 28322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:35.946561 28322 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [term 1 LEADER]: Becoming Leader. State: Replica: b2dfbb45d52e4cc7b03c41b8bb870272, State: Running, Role: LEADER
I20260811 02:51:35.946956 28322 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2dfbb45d52e4cc7b03c41b8bb870272" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36421 } }
I20260811 02:51:35.947535 28309 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:35.949121 28328 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2dfbb45d52e4cc7b03c41b8bb870272. Latest consensus state: current_term: 1 leader_uuid: "b2dfbb45d52e4cc7b03c41b8bb870272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2dfbb45d52e4cc7b03c41b8bb870272" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36421 } } }
I20260811 02:51:35.949637 28328 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:35.949522 28325 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2dfbb45d52e4cc7b03c41b8bb870272" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2dfbb45d52e4cc7b03c41b8bb870272" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 36421 } } }
I20260811 02:51:35.949970 28325 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272 [sys.catalog]: This master's current role is: LEADER
W20260811 02:51:35.953011 28355 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:51:35.953116 28355 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:51:35.953205 28356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:35.954049 28356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:35.960035 28356 catalog_manager.cc:1347] Generated new cluster ID: e921e80ba3224bf7aa5cb3d7e602086b
I20260811 02:51:35.960147 28356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:35.968281 28356 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:35.968643 28356 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:35.981154 28356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2dfbb45d52e4cc7b03c41b8bb870272: Generated new TSK 0
I20260811 02:51:35.981956 28356 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 02:51:35.993175 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 28193
2026-08-11T02:51:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1726 ms)
[----------] 18 tests from AuthTokenIssuingTest (31406 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:36 dist-test-slave-n326 krb5kdc[28363](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:36 dist-test-slave-n326 krb5kdc[28363](info): set up 2 sockets
Aug 11 02:51:36 dist-test-slave-n326 krb5kdc[28363](info): commencing operation
krb5kdc: starting...
W20260811 02:51:37.345331 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:37 dist-test-slave-n326 krb5kdc[28363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:51:37.496313 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:33965
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:42233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:33965,127.21.236.189:45999,127.21.236.188:45765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:37.614436 28379 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 02:51:37.614758 28379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:37.614784 28379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:37.614823 28379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:37.618891 28379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:37.618966 28379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:37.618988 28379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:37.619004 28379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:37.619019 28379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:37.624392 28379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:42233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:33965,127.21.236.189:45999,127.21.236.188:45765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:33965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:37.625417 28379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:37.626498 28379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:37.635532 28385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:37.635550 28392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:37.635524 28388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:37.635881 28379 server_base.cc:1034] running on GCE node
I20260811 02:51:37.636583 28379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:37.637877 28379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:37.639163 28379 hybrid_clock.cc:648] HybridClock initialized: now 1786416697639141 us; error 35 us; skew 500 ppm
Aug 11 02:51:37 dist-test-slave-n326 krb5kdc[28363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:37.642345 28379 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:51:37.643592 28379 webserver.cc:460] Webserver started at http://127.21.236.190:41343/ using document root <none> and password file <none>
I20260811 02:51:37.644243 28379 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:37.644351 28379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:37.644629 28379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:37.646950 28379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "a9a5dc2074934229b05205314df81909"
format_stamp: "Formatted at 2026-08-11 02:51:37 on dist-test-slave-n326"
server_key: "5ceefbc3531245b80998de3b89fd0f84"
server_key_iv: "d7e3da0d4b6325d0d3b0ff85adf425d3"
server_key_version: "encryptionkey@0"
I20260811 02:51:37.647665 28379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "a9a5dc2074934229b05205314df81909"
format_stamp: "Formatted at 2026-08-11 02:51:37 on dist-test-slave-n326"
server_key: "5ceefbc3531245b80998de3b89fd0f84"
server_key_iv: "d7e3da0d4b6325d0d3b0ff85adf425d3"
server_key_version: "encryptionkey@0"
I20260811 02:51:37.651854 28379 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:51:37.654796 28407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:37.656262 28379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:51:37.656451 28379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "a9a5dc2074934229b05205314df81909"
format_stamp: "Formatted at 2026-08-11 02:51:37 on dist-test-slave-n326"
server_key: "5ceefbc3531245b80998de3b89fd0f84"
server_key_iv: "d7e3da0d4b6325d0d3b0ff85adf425d3"
server_key_version: "encryptionkey@0"
I20260811 02:51:37.656584 28379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:37.678148 28379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:37.681752 28379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:37.681988 28379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:37.689896 28379 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:33965
I20260811 02:51:37.690224 28507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:33965 every 8 connection(s)
I20260811 02:51:37.690968 28379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:37.693915 28509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:37.694734 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 28379
I20260811 02:51:37.694840 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:37.695253 22450 external_mini_cluster.cc:1428] Setting key 76c4d1e979386f9223b2f411a3d725ae
I20260811 02:51:37.698066 28509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.236.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.189@KRBTEST.COM" created.
Aug 11 02:51:37 dist-test-slave-n326 krb5kdc[28363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:51:37.714759 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.701976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49142 (local address 127.21.236.190:33965)
0811 02:51:37.702334 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:37.702346 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:37.702389 (+ 43us) server_negotiation.cc:407] Connection header received
0811 02:51:37.702758 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:37.702762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:37.702940 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:37.703179 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:37.704029 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:37.704907 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:37.705978 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:37.706349 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:37.706525 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:37.709580 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:37.709587 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:37.709597 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:37.709619 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:37.711487 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:37.711977 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:37.711983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:37.711985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:37.712096 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:37.712446 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:37.712449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:37.712451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:37.712854 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:37.713012 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:37.713388 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 02:51:37.713661 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":118,"threads_started":1}
I20260811 02:51:37.714759 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.700827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:33965 (local address 127.0.0.1:49142)
0811 02:51:37.701442 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:37.701485 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:37.702453 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:37.703180 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:37.703186 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:37.703625 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:37.703874 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:37.703888 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:37.705047 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:37.705055 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:37.705852 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:37.705862 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:37.706507 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:37.706511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:37.706621 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:37.707380 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:37.707398 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:37.709396 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:37.711599 (+ 2203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:37.711610 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:37.711621 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:37.711854 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:37.712200 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:37.712203 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:37.712205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:37.712325 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:37.712983 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:37.712990 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:37.713139 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 02:51:37.713318 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 02:51:37.713741 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":107,"thread_start_us":182,"threads_started":1}
I20260811 02:51:37.717163 28509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:51:37.718051 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.717702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:45999 (local address 127.0.0.1:49294)
0811 02:51:37.717905 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:37.717982 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 02:51:37.718293 28412 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.236.189:45999: connect: Connection refused (error 111)
I20260811 02:51:37.719408 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.719005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:45999 (local address 127.0.0.1:49304)
0811 02:51:37.719242 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:37.719309 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260811 02:51:37.719604 28509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.189:45999: Network error: Client connection negotiation failed: client connection to 127.21.236.189:45999: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.189.keytab.
Entry for principal kudu/127.21.236.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.189.keytab.
I20260811 02:51:37.727813 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.189:45999
--webserver_interface=127.21.236.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.189.keytab
--principal=kudu/127.21.236.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:42233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:33965,127.21.236.189:45999,127.21.236.188:45765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:51:37.768937 28509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } attempt: 1
I20260811 02:51:37.769996 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.769596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:45999 (local address 127.0.0.1:49318)
0811 02:51:37.769801 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:37.769891 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 02:51:37.770875 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.770608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:45999 (local address 127.0.0.1:49330)
0811 02:51:37.770737 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:37.770799 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 02:51:37.771160 28509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.189:45999: Network error: Client connection negotiation failed: client connection to 127.21.236.189:45999: connect: Connection refused (error 111)
I20260811 02:51:37.839406 28509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } attempt: 2
I20260811 02:51:37.840394 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.840027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:45999 (local address 127.0.0.1:49340)
0811 02:51:37.840218 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:37.840297 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 02:51:37.841266 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.840990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:45999 (local address 127.0.0.1:49352)
0811 02:51:37.841135 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:37.841193 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 02:51:37.841454 28509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.189:45999: Network error: Client connection negotiation failed: client connection to 127.21.236.189:45999: connect: Connection refused (error 111)
W20260811 02:51:37.854904 28523 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 02:51:37.855432 28523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:37.855485 28523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:37.855527 28523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:37.859686 28523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:37.859750 28523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:37.859771 28523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:37.859788 28523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:37.859803 28523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:37.865417 28523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:42233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:33965,127.21.236.189:45999,127.21.236.188:45765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.189.keytab
--principal=kudu/127.21.236.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.189:45999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.236.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:37.866652 28523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:37.867785 28523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:37.876940 28529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:37.877022 28523 server_base.cc:1034] running on GCE node
W20260811 02:51:37.876959 28533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:37.876976 28530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:37.877873 28523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:37.879045 28523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:37.880312 28523 hybrid_clock.cc:648] HybridClock initialized: now 1786416697880278 us; error 48 us; skew 500 ppm
Aug 11 02:51:37 dist-test-slave-n326 krb5kdc[28363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:37.883425 28523 init.cc:377] Logged in from keytab as kudu/127.21.236.189@KRBTEST.COM (short username kudu)
I20260811 02:51:37.884518 28523 webserver.cc:460] Webserver started at http://127.21.236.189:41523/ using document root <none> and password file <none>
I20260811 02:51:37.885218 28523 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:37.885334 28523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:37.885618 28523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:37.888041 28523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/data/instance:
uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a"
format_stamp: "Formatted at 2026-08-11 02:51:37 on dist-test-slave-n326"
server_key: "016171cc460bbde6092ccfda7a3552e7"
server_key_iv: "2a7bd59be537354486c8c6ca6d6e014f"
server_key_version: "encryptionkey@0"
I20260811 02:51:37.888715 28523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/wal/instance:
uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a"
format_stamp: "Formatted at 2026-08-11 02:51:37 on dist-test-slave-n326"
server_key: "016171cc460bbde6092ccfda7a3552e7"
server_key_iv: "2a7bd59be537354486c8c6ca6d6e014f"
server_key_version: "encryptionkey@0"
I20260811 02:51:37.893137 28523 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 02:51:37.896157 28548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:37.897859 28523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 02:51:37.898056 28523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/wal
uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a"
format_stamp: "Formatted at 2026-08-11 02:51:37 on dist-test-slave-n326"
server_key: "016171cc460bbde6092ccfda7a3552e7"
server_key_iv: "2a7bd59be537354486c8c6ca6d6e014f"
server_key_version: "encryptionkey@0"
I20260811 02:51:37.898209 28523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:37.925292 28523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:37.928966 28523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:37.929208 28523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:37.932646 28509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } attempt: 3
I20260811 02:51:37.933612 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.933202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:45999 (local address 127.0.0.1:49368)
0811 02:51:37.933409 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:37.933509 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 02:51:37.934554 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.934294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:45999 (local address 127.0.0.1:49382)
0811 02:51:37.934431 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:37.934491 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:45999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 02:51:37.934712 28509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.189:45999: Network error: Client connection negotiation failed: client connection to 127.21.236.189:45999: connect: Connection refused (error 111)
I20260811 02:51:37.937556 28523 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.189:45999
I20260811 02:51:37.937861 28654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.189:45999 every 8 connection(s)
I20260811 02:51:37.938728 28523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/data/info.pb
I20260811 02:51:37.941660 28657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:37.945626 28657 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:51:37.947232 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 28523
I20260811 02:51:37.947362 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-1/wal/instance
I20260811 02:51:37.947690 22450 external_mini_cluster.cc:1428] Setting key 2b4b5be66c2197cc2306e5f0501f78cd
Aug 11 02:51:37 dist-test-slave-n326 krb5kdc[28363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.189@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.236.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.188@KRBTEST.COM" created.
I20260811 02:51:37.961593 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.947785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.21.236.190:33965)
0811 02:51:37.947976 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:37.947981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:37.949153 (+ 1172us) server_negotiation.cc:407] Connection header received
0811 02:51:37.950187 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:37.950191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:37.950258 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:37.950337 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:37.952001 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:37.952545 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:37.953735 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:37.954009 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:37.954135 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:37.957051 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:37.957054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:37.957056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:37.957085 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:37.958783 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:37.959587 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:37.959591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:37.959593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:37.959682 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:37.960099 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:37.960103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:37.960104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:37.960375 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:37.960488 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:37.961147 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 02:51:37.961308 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 02:51:37.962232 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.948252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:33965 (local address 127.0.0.1:49158)
0811 02:51:37.948859 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:37.949000 (+ 141us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:37.949950 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:37.950509 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:37.950517 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:37.950873 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:37.951707 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:37.951721 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:37.952689 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:37.952693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:37.953592 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:37.953602 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:37.954180 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:37.954184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:37.954370 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:37.955099 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:37.955143 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:37.956817 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:37.958954 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:37.958961 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:37.958979 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:37.959470 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:37.959786 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:37.959790 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:37.959792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:37.959979 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:37.960498 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:37.960508 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:37.960797 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 02:51:37.961000 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 02:51:37.961308 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":165,"threads_started":1}
I20260811 02:51:37.963932 28657 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:51:37 dist-test-slave-n326 krb5kdc[28363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.189@KRBTEST.COM for kudu/127.21.236.189@KRBTEST.COM
I20260811 02:51:37.974063 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.964655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:45999 (local address 127.0.0.1:49396)
0811 02:51:37.964795 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:37.964809 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:37.964891 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:37.965489 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:37.965492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:37.965685 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:37.965772 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:37.965777 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:37.966769 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:37.966775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:37.967396 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:37.967405 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:37.968042 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:37.968045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:37.968211 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:37.968741 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:37.968756 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:37.970038 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:37.972304 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:37.972308 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:37.972310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:37.972611 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:37.972925 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:37.972928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:37.972930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:37.973014 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:37.973588 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:37.973591 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:37.973680 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 02:51:37.973796 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 02:51:37.973923 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":47}
I20260811 02:51:37.974327 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.964762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49396 (local address 127.21.236.189:45999)
0811 02:51:37.965149 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:37.965157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:37.965176 (+ 19us) server_negotiation.cc:407] Connection header received
0811 02:51:37.965210 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:37.965215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:37.965364 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:37.965524 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:37.965886 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:37.966643 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:37.967517 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:37.967913 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:37.968044 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:37.970197 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:37.970208 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:37.970224 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:37.970251 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:37.972169 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:37.972723 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:37.972729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:37.972731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:37.972817 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:37.973111 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:37.973114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:37.973116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:37.973428 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:37.973579 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:37.973896 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 02:51:37.974055 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":300,"thread_start_us":189,"threads_started":1}
I20260811 02:51:37.976384 28657 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:51:37.977203 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.976925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:45765 (local address 127.0.0.1:51748)
0811 02:51:37.977055 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:37.977141 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 02:51:37.977451 28556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
I20260811 02:51:37.978504 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:37.978207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:45765 (local address 127.0.0.1:51756)
0811 02:51:37.978311 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:37.978417 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 02:51:37.978672 28657 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:45765: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.188.keytab.
Entry for principal kudu/127.21.236.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.188.keytab.
I20260811 02:51:37.982228 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.188:45765
--webserver_interface=127.21.236.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.188.keytab
--principal=kudu/127.21.236.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:42233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:33965,127.21.236.189:45999,127.21.236.188:45765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:51:38.027961 28657 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } attempt: 1
I20260811 02:51:38.028944 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.028617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:45765 (local address 127.0.0.1:51766)
0811 02:51:38.028780 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:38.028857 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 02:51:38.029805 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.029576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:45765 (local address 127.0.0.1:51780)
0811 02:51:38.029693 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:38.029744 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 02:51:38.030097 28657 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:45765: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
I20260811 02:51:38.077877 28509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } attempt: 4
Aug 11 02:51:38 dist-test-slave-n326 krb5kdc[28363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for kudu/127.21.236.189@KRBTEST.COM
I20260811 02:51:38.088608 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.078563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:45999 (local address 127.0.0.1:49398)
0811 02:51:38.078800 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:38.078821 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:38.078960 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:38.079319 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:38.079322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:38.079500 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:38.079974 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:38.079982 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.081506 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.081509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:38.082068 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:38.082078 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.082758 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.082761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:38.082851 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.083454 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:38.083467 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:38.084844 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:38.086974 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:38.086977 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:38.086978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:38.087276 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:38.087606 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:38.087608 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:38.087610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:38.087695 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:38.088157 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:38.088159 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:38.088251 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 02:51:38.088340 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 02:51:38.088469 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260811 02:51:38.088754 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.078710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49398 (local address 127.21.236.189:45999)
0811 02:51:38.078900 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.078905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.078925 (+ 20us) server_negotiation.cc:407] Connection header received
0811 02:51:38.079118 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.079123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.079183 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.079281 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.080130 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.081338 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.082206 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.082619 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.082751 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.085003 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:38.085007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:38.085012 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:38.085043 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:38.086845 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.087383 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.087386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.087388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.087502 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.087803 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.087807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.087808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.088053 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:38.088158 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:38.088437 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 02:51:38.088585 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:51:38.089551 28509 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:51:38.090312 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.090036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:45765 (local address 127.0.0.1:51794)
0811 02:51:38.090176 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:38.090239 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 02:51:38.091135 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.090875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:45765 (local address 127.0.0.1:51806)
0811 02:51:38.090998 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:38.091050 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 02:51:38.091274 28509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:45765: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
I20260811 02:51:38.098317 28657 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } attempt: 2
I20260811 02:51:38.099244 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.098904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:45765 (local address 127.0.0.1:51812)
0811 02:51:38.099057 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:38.099153 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 02:51:38.100111 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.099832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:45765 (local address 127.0.0.1:51816)
0811 02:51:38.100000 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:38.100056 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260811 02:51:38.100240 28657 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:45765: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
W20260811 02:51:38.108951 28669 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 02:51:38.109310 28669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:38.109378 28669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:38.109454 28669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:38.113610 28669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:38.113704 28669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:38.113752 28669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:38.113806 28669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:38.113842 28669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:38.119333 28669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:42233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:33965,127.21.236.189:45999,127.21.236.188:45765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.188.keytab
--principal=kudu/127.21.236.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.188:45765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.236.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:38.120493 28669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:38.121567 28669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:38.131419 28678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:38.131444 28682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:38.131431 28669 server_base.cc:1034] running on GCE node
W20260811 02:51:38.131435 28679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:38.132364 28669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:38.133472 28669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:38.134634 28669 hybrid_clock.cc:648] HybridClock initialized: now 1786416698134594 us; error 54 us; skew 500 ppm
Aug 11 02:51:38 dist-test-slave-n326 krb5kdc[28363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:38.137629 28669 init.cc:377] Logged in from keytab as kudu/127.21.236.188@KRBTEST.COM (short username kudu)
I20260811 02:51:38.138561 28669 webserver.cc:460] Webserver started at http://127.21.236.188:46577/ using document root <none> and password file <none>
I20260811 02:51:38.139227 28669 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:38.139298 28669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:38.139523 28669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:38.141628 28669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/data/instance:
uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33"
format_stamp: "Formatted at 2026-08-11 02:51:38 on dist-test-slave-n326"
server_key: "a0229fd3f7aecc8b16891aedc10ef24a"
server_key_iv: "0371fbf3496fa50167fb4935ade620c4"
server_key_version: "encryptionkey@0"
I20260811 02:51:38.142242 28669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/wal/instance:
uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33"
format_stamp: "Formatted at 2026-08-11 02:51:38 on dist-test-slave-n326"
server_key: "a0229fd3f7aecc8b16891aedc10ef24a"
server_key_iv: "0371fbf3496fa50167fb4935ade620c4"
server_key_version: "encryptionkey@0"
I20260811 02:51:38.146307 28669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 02:51:38.149122 28692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:38.150446 28509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } attempt: 1
I20260811 02:51:38.150543 28669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 02:51:38.150702 28669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/wal
uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33"
format_stamp: "Formatted at 2026-08-11 02:51:38 on dist-test-slave-n326"
server_key: "a0229fd3f7aecc8b16891aedc10ef24a"
server_key_iv: "0371fbf3496fa50167fb4935ade620c4"
server_key_version: "encryptionkey@0"
I20260811 02:51:38.150816 28669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:38.151357 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.151049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:45765 (local address 127.0.0.1:51820)
0811 02:51:38.151217 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:38.151281 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 02:51:38.152097 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.151870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:45765 (local address 127.0.0.1:51828)
0811 02:51:38.151986 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:38.152041 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 02:51:38.152244 28509 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:45765: Network error: Client connection negotiation failed: client connection to 127.21.236.188:45765: connect: Connection refused (error 111)
I20260811 02:51:38.169656 28669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:38.173144 28669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:38.173378 28669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:38.181133 28669 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.188:45765
I20260811 02:51:38.181442 28779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.188:45765 every 8 connection(s)
I20260811 02:51:38.182168 28669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/data/info.pb
I20260811 02:51:38.185101 28781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:38.188900 28781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:51:38.191195 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 28669
I20260811 02:51:38.191340 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786416395682572-22450-0/minicluster-data/master-2/wal/instance
I20260811 02:51:38.191401 28657 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } attempt: 3
I20260811 02:51:38.191681 22450 external_mini_cluster.cc:1428] Setting key 8a08b5f9dd84e6a13ca330c7eb24d860
Aug 11 02:51:38 dist-test-slave-n326 krb5kdc[28363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.189@KRBTEST.COM for kudu/127.21.236.188@KRBTEST.COM
Aug 11 02:51:38 dist-test-slave-n326 krb5kdc[28363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.188@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
Aug 11 02:51:38 dist-test-slave-n326 krb5kdc[28363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.188@KRBTEST.COM
I20260811 02:51:38.205082 28661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.191866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:45765 (local address 127.0.0.1:51842)
0811 02:51:38.192163 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:38.192177 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:38.192273 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:38.193665 (+ 1392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:38.193668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:38.193896 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:38.194029 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:38.194036 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.195668 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.195672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:38.196511 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:38.196526 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.197194 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.197197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:38.197290 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.198053 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:38.198071 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:38.199891 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:38.202781 (+ 2890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:38.202786 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:38.202788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:38.203152 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:38.203627 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:38.203631 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:38.203633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:38.203730 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:38.204525 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:38.204528 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:38.204625 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 02:51:38.204712 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 02:51:38.204887 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223}
I20260811 02:51:38.205155 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.190882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49160 (local address 127.21.236.190:33965)
0811 02:51:38.191051 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.191056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.192068 (+ 1012us) server_negotiation.cc:407] Connection header received
0811 02:51:38.193472 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.193478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.193542 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.193636 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.194686 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.195478 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.196844 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.197180 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.197271 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.200987 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:38.200993 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:38.200998 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:38.201039 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:38.202826 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.203542 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.203549 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.203554 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.203655 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.204049 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.204053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.204054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.204287 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:38.204375 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:38.204889 (+ 514us) server_negotiation.cc:299] Negotiation successful
0811 02:51:38.205011 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 02:51:38.206807 28784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.191290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:33965 (local address 127.0.0.1:49160)
0811 02:51:38.191899 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:38.191961 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:38.193209 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:38.193818 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:38.193826 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:38.194323 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:38.194534 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:38.194547 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.195598 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.195601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:38.196724 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:38.196731 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.197421 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.197430 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:38.197591 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.198239 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:38.198255 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:38.200749 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:38.202954 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:38.202966 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:38.202980 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:38.203361 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:38.203780 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:38.203784 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:38.203788 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:38.203933 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:38.204442 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:38.204452 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:38.204629 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 02:51:38.205308 (+ 679us) client_negotiation.cc:240] Negotiation successful
0811 02:51:38.205711 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":22,"thread_start_us":143,"threads_started":1}
I20260811 02:51:38.206817 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.192327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51842 (local address 127.21.236.188:45765)
0811 02:51:38.192761 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.192771 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.192812 (+ 41us) server_negotiation.cc:407] Connection header received
0811 02:51:38.193231 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.193236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.193459 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.194148 (+ 689us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.194265 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.195488 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.196693 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.197032 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.197596 (+ 564us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.200115 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:38.200123 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:38.200141 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:38.200171 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:38.202624 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.203348 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.203354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.203356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.203514 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.203843 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.203846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.203848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.204403 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:38.205906 (+ 1503us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:38.206027 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 02:51:38.206173 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":348,"thread_start_us":168,"threads_started":1}
I20260811 02:51:38.207587 28786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.192486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51850 (local address 127.21.236.188:45765)
0811 02:51:38.192881 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.192890 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.192906 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:51:38.193209 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.193233 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.193458 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.193660 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.194487 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.195577 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.196418 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.197016 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.197190 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.202532 (+ 5342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:38.202536 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:38.202540 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:38.202586 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:38.205161 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.205726 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.205729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.205731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.205822 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.206146 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.206150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.206152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.206360 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:38.206542 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:38.207271 (+ 729us) server_negotiation.cc:299] Negotiation successful
0811 02:51:38.207393 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":130,"server-negotiator.queue_time_us":326,"thread_start_us":77,"threads_started":1}
I20260811 02:51:38.208428 28781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:51:38.211953 28657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a: Bootstrap starting.
Aug 11 02:51:38 dist-test-slave-n326 krb5kdc[28363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.188@KRBTEST.COM for kudu/127.21.236.189@KRBTEST.COM
I20260811 02:51:38.215011 28657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a: Neither blocks nor log segments found. Creating new log.
Aug 11 02:51:38 dist-test-slave-n326 krb5kdc[28363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:51:38.216094 28657 log.cc:824] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:38.218850 28657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a: No bootstrap required, opened a new log
I20260811 02:51:38.218987 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.209085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49414 (local address 127.21.236.189:45999)
0811 02:51:38.209194 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.209198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.209379 (+ 181us) server_negotiation.cc:407] Connection header received
0811 02:51:38.209490 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.209493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.209541 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.209632 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.210149 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.210900 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.211765 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.212044 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.212141 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.215296 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:38.215300 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:38.215303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:38.215329 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:38.217202 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.217644 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.217648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.217650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.217731 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.218021 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.218024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.218026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.218319 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:38.218426 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:38.218726 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 02:51:38.218824 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 02:51:38.219026 28784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.208939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:45999 (local address 127.0.0.1:49414)
0811 02:51:38.209310 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:38.209325 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:38.209411 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:38.209646 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:38.209649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:38.209849 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:38.209993 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:38.210002 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.211049 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.211052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:38.211632 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:38.211641 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.212265 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.212269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:38.212373 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.213415 (+ 1042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:38.213433 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:38.215127 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:38.217320 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:38.217324 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:38.217326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:38.217529 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:38.217831 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:38.217835 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:38.217837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:38.217924 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:38.218438 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:38.218441 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:38.218551 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 02:51:38.218672 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 02:51:38.218817 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303}
I20260811 02:51:38.219395 28509 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } attempt: 2
I20260811 02:51:38.219748 28781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:51:38.220814 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.210084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49176 (local address 127.21.236.190:33965)
0811 02:51:38.210189 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.210192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.210204 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:51:38.210241 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.210244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.210289 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.210371 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.211159 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.211873 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.212927 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.213161 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.213280 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.216189 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:38.216194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:38.216198 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:38.216231 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:38.217982 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.219209 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.219213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.219215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.219313 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.219652 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.219658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.219662 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.219922 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:38.220133 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:38.220528 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 02:51:38.220675 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 02:51:38 dist-test-slave-n326 krb5kdc[28363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416698, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.188@KRBTEST.COM for kudu/127.21.236.188@KRBTEST.COM
I20260811 02:51:38.225705 28657 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } }
Aug 11 02:51:38 dist-test-slave-n326 krb5kdc[28363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416697, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for kudu/127.21.236.188@KRBTEST.COM
I20260811 02:51:38.225975 28657 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c8d1dcc0e94b67b3a9510461cadc8a, State: Initialized, Role: FOLLOWER
I20260811 02:51:38.226418 28657 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } }
I20260811 02:51:38.227020 28789 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } } }
I20260811 02:51:38.227209 28789 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:51:38.228060 28657 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 02:51:38 dist-test-slave-n326 krb5kdc[28363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.189@KRBTEST.COM
I20260811 02:51:38.230638 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.219794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:45765 (local address 127.0.0.1:51864)
0811 02:51:38.220068 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:38.220082 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:38.220156 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:38.220590 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:38.220594 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:38.220813 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:38.220916 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:38.220923 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.221991 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.221994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:38.222842 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:38.222852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.223571 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.223575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:38.223657 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.224339 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:38.224355 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:38.226682 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:38.228778 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:38.228785 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:38.228788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:38.229085 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:38.229416 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:38.229421 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:38.229424 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:38.229524 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:38.230029 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:38.230035 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:38.230164 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 02:51:38.230271 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 02:51:38.230439 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260811 02:51:38.231156 28784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.220227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:45765 (local address 127.0.0.1:51874)
0811 02:51:38.220698 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:38.220712 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:38.220780 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:38.221012 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:38.221014 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:38.221176 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:38.221251 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:38.221257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.222228 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.222232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:38.222997 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:38.223004 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.223603 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.223607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:38.223675 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.224269 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:38.224283 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:38.226316 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:38.229651 (+ 3335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:38.229654 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:38.229655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:38.229889 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:38.230160 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:38.230163 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:38.230165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:38.230243 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:38.230627 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:38.230629 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:38.230717 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 02:51:38.230800 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 02:51:38.230951 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411}
I20260811 02:51:38.231529 28786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.220299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51864 (local address 127.21.236.188:45765)
0811 02:51:38.220386 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.220390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.220400 (+ 10us) server_negotiation.cc:407] Connection header received
0811 02:51:38.220429 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.220432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.220472 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.220564 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.221036 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.221868 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.222971 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.223345 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.223436 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.226842 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:38.226848 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:38.226851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:38.226879 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:38.228678 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.229211 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.229214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.229216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.229303 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.229631 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.229637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.229641 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.229889 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:38.229980 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:38.231255 (+ 1275us) server_negotiation.cc:299] Negotiation successful
0811 02:51:38.231387 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":34}
I20260811 02:51:38.234279 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.222523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49426 (local address 127.21.236.189:45999)
0811 02:51:38.222621 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.222626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.222776 (+ 150us) server_negotiation.cc:407] Connection header received
0811 02:51:38.222970 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.222974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.223017 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.223112 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.223984 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.224701 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.225882 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.226197 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.226303 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.230057 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:38.230064 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:38.230070 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:38.230100 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:38.231834 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.232601 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.232605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.232607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.232702 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.233049 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.233052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.233054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.233257 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:38.233521 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:38.233975 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 02:51:38.234136 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 02:51:38.235332 28509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909: Bootstrap starting.
I20260811 02:51:38.232939 28523 master_runner.cc:186] Error getting master registration for 127.21.236.190:33965: OK, instance_id { permanent_uuid: "a9a5dc2074934229b05205314df81909" instance_seqno: 1786416697687769 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 02:51:38.236238 28523 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:51:38.236701 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.220251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51874 (local address 127.21.236.188:45765)
0811 02:51:38.220350 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.220354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.220849 (+ 495us) server_negotiation.cc:407] Connection header received
0811 02:51:38.220873 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.220875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.220912 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.221346 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.221425 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.222131 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.223153 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.223469 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.223865 (+ 396us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.226447 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:38.226451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:38.226453 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:38.226476 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:38.229539 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.229988 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.229993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.229997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.230068 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.230338 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.230340 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.230342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.230518 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:38.236407 (+ 5889us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:38.236481 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 02:51:38.236594 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 02:51:38.238186 28509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:38.238962 28509 log.cc:824] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:38.239370 28781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33: Bootstrap starting.
I20260811 02:51:38.241312 28509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909: No bootstrap required, opened a new log
W20260811 02:51:38.241752 28821 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:51:38.241945 28821 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:51:38.242108 28781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:38.243010 28781 log.cc:824] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:38.244674 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:38.245321 28781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33: No bootstrap required, opened a new log
I20260811 02:51:38.245419 28509 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } }
I20260811 02:51:38.245651 28509 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9a5dc2074934229b05205314df81909, State: Initialized, Role: FOLLOWER
I20260811 02:51:38.245980 28509 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } }
I20260811 02:51:38.246207 28457 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a9a5dc2074934229b05205314df81909"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49176
I20260811 02:51:38.247495 28509 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:38.247586 28457 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 02:51:38.247731 28457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:38.249034 28457 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } }
I20260811 02:51:38.249603 28781 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } }
I20260811 02:51:38.249989 28457 leader_election.cc:288] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [CANDIDATE]: Term 1 election: Requested vote from peers c4c8d1dcc0e94b67b3a9510461cadc8a (127.21.236.189:45999), c1bbd6cd02ff4a7fbee8e9a02bb1ae33 (127.21.236.188:45765)
I20260811 02:51:38.250363 28781 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1bbd6cd02ff4a7fbee8e9a02bb1ae33, State: Initialized, Role: FOLLOWER
I20260811 02:51:38.250835 28781 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } }
I20260811 02:51:38.251614 28610 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9a5dc2074934229b05205314df81909" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a"
I20260811 02:51:38.251865 28610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:38.252507 28781 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:38.252578 28741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9a5dc2074934229b05205314df81909" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33"
I20260811 02:51:38.252761 28741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:38.253266 28610 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a5dc2074934229b05205314df81909 in term 1.
I20260811 02:51:38.254361 28741 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9a5dc2074934229b05205314df81909 in term 1.
I20260811 02:51:38.254632 28412 leader_election.cc:302] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9a5dc2074934229b05205314df81909, c4c8d1dcc0e94b67b3a9510461cadc8a; no voters:
I20260811 02:51:38.255151 28863 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } } }
I20260811 02:51:38.255282 28863 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:51:38.255383 28863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:38.255458 28863 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [term 1 LEADER]: Becoming Leader. State: Replica: a9a5dc2074934229b05205314df81909, State: Running, Role: LEADER
I20260811 02:51:38.256379 28863 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } }
I20260811 02:51:38.259218 28826 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9a5dc2074934229b05205314df81909. Latest consensus state: current_term: 1 leader_uuid: "a9a5dc2074934229b05205314df81909" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } } }
I20260811 02:51:38.259341 28826 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:38.260134 28876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:38.262923 28379 master_runner.cc:263] No leader master found from master a9a5dc2074934229b05205314df81909
I20260811 02:51:38.262992 28379 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 02:51:38.266417 28867 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:51:38.266517 28867 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:51:38.267407 28847 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } } }
I20260811 02:51:38.267549 28847 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:51:38.268165 28876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:38.274349 28667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.260470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49440 (local address 127.21.236.189:45999)
0811 02:51:38.260578 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.260583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.263391 (+ 2808us) server_negotiation.cc:407] Connection header received
0811 02:51:38.263908 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.263912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.263961 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.264036 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.265115 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.265890 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.267921 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.268248 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.268330 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.270302 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:38.270306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:38.270308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:38.270336 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:38.272345 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.272846 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.272850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.272852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.272943 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.273281 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.273285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.273287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.273516 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:38.273626 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:38.274044 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 02:51:38.274188 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 02:51:38.275681 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.257572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49190 (local address 127.21.236.190:33965)
0811 02:51:38.258083 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.258088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.261953 (+ 3865us) server_negotiation.cc:407] Connection header received
0811 02:51:38.262152 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.262156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.262203 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.262252 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.267320 (+ 5068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.268096 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.269275 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.269646 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.269730 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.271808 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:38.271812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:38.271815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:38.271843 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:38.273542 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.274214 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.274217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.274219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.274316 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.274639 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.274642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.274644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.274861 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:38.274995 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:38.275397 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 02:51:38.275525 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421}
I20260811 02:51:38.284787 28741 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33 [term 1 FOLLOWER]: Refusing update from remote peer a9a5dc2074934229b05205314df81909: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:38.285589 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:38.261335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51886 (local address 127.21.236.188:45765)
0811 02:51:38.261480 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.261484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.276493 (+ 15009us) server_negotiation.cc:407] Connection header received
0811 02:51:38.276667 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.276670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.276720 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.276773 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.277658 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.278520 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.279611 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:38.279907 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:38.279984 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:38.281710 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:38.281714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:38.281717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:38.281746 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:38.283617 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.284140 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.284144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.284146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.284235 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:38.284538 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:38.284544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:38.284547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:38.284765 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:38.284891 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:38.285302 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 02:51:38.285431 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 02:51:38.285681 28610 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a [term 1 FOLLOWER]: Refusing update from remote peer a9a5dc2074934229b05205314df81909: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:38.286774 28826 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:38.287194 28863 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:38.291153 28789 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9a5dc2074934229b05205314df81909. Latest consensus state: current_term: 1 leader_uuid: "a9a5dc2074934229b05205314df81909" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } } }
I20260811 02:51:38.291307 28789 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:51:38.296317 28847 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9a5dc2074934229b05205314df81909. Latest consensus state: current_term: 1 leader_uuid: "a9a5dc2074934229b05205314df81909" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } } }
I20260811 02:51:38.296471 28847 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:51:38.308128 28863 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9a5dc2074934229b05205314df81909" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } } }
I20260811 02:51:38.308317 28863 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:38.308454 28891 mvcc.cc:205] Tried to move back new op lower bound from 7317162796159811584 to 7317162796061716480. Current Snapshot: MvccSnapshot[applied={T|T < 7317162796159811584}]
I20260811 02:51:38.312266 28847 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9a5dc2074934229b05205314df81909" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } } }
I20260811 02:51:38.312445 28847 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:51:38.313148 28898 mvcc.cc:205] Tried to move back new op lower bound from 7317162796159811584 to 7317162796061716480. Current Snapshot: MvccSnapshot[applied={T|T < 7317162796159811584}]
I20260811 02:51:38.313937 28826 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9a5dc2074934229b05205314df81909" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } } }
I20260811 02:51:38.314076 28826 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909 [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:38.315482 28789 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9a5dc2074934229b05205314df81909" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9a5dc2074934229b05205314df81909" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 33965 } } peers { permanent_uuid: "c4c8d1dcc0e94b67b3a9510461cadc8a" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 45999 } } peers { permanent_uuid: "c1bbd6cd02ff4a7fbee8e9a02bb1ae33" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 45765 } } }
I20260811 02:51:38.315614 28789 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:51:38.316382 28892 mvcc.cc:205] Tried to move back new op lower bound from 7317162796159811584 to 7317162796061716480. Current Snapshot: MvccSnapshot[applied={T|T < 7317162796159811584}]
I20260811 02:51:38.322842 28876 catalog_manager.cc:1347] Generated new cluster ID: 2e23cb8f9cb0415cb60e1f6136ed8314
I20260811 02:51:38.322937 28876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:38.356427 28876 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:38.357743 28876 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:38.378121 28876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9a5dc2074934229b05205314df81909: Generated new TSK 0
I20260811 02:51:38.379282 28876 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 02:51:38.410005 28667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:38.406668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49452 (local address 127.21.236.189:45999)
0811 02:51:38.406798 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.406803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.408792 (+ 1989us) server_negotiation.cc:407] Connection header received
0811 02:51:38.409006 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.409010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.409063 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.409122 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.409843 (+ 721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49452: BlockingRecv error: recv got EOF from 127.0.0.1:49452 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 02:51:38.411963 28787 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:38.407390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51890 (local address 127.21.236.188:45765)
0811 02:51:38.407546 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.407551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.410723 (+ 3172us) server_negotiation.cc:407] Connection header received
0811 02:51:38.411148 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.411151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.411206 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.411261 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.411816 (+ 555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51890: BlockingRecv error: recv got EOF from 127.0.0.1:51890 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 02:51:38.413311 28517 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:38.408043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49192 (local address 127.21.236.190:33965)
0811 02:51:38.408184 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.408190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.410935 (+ 2745us) server_negotiation.cc:407] Connection header received
0811 02:51:38.412497 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.412500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.412556 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.412612 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.413186 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49192: BlockingRecv error: recv got EOF from 127.0.0.1:49192 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 02:51:38.418881 28667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:38.417176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49458 (local address 127.21.236.189:45999)
0811 02:51:38.417307 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.417312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.417777 (+ 465us) server_negotiation.cc:407] Connection header received
0811 02:51:38.418042 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.418046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.418127 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.418185 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.418761 (+ 576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49458: BlockingRecv error: recv got EOF from 127.0.0.1:49458 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 02:51:38.425086 28667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:38.423107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49462 (local address 127.21.236.189:45999)
0811 02:51:38.423236 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.423241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.423939 (+ 698us) server_negotiation.cc:407] Connection header received
0811 02:51:38.424188 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.424193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.424255 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.424315 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.424958 (+ 643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49462: BlockingRecv error: recv got EOF from 127.0.0.1:49462 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 02:51:38.431886 28667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:38.429674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49470 (local address 127.21.236.189:45999)
0811 02:51:38.429791 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.429796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.430616 (+ 820us) server_negotiation.cc:407] Connection header received
0811 02:51:38.430909 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.430915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.430986 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.431054 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.431748 (+ 694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49470: BlockingRecv error: recv got EOF from 127.0.0.1:49470 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 02:51:38.440761 28667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:38.438668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49484 (local address 127.21.236.189:45999)
0811 02:51:38.438843 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.438848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.439586 (+ 738us) server_negotiation.cc:407] Connection header received
0811 02:51:38.439839 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.439844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.439922 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.439990 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.440634 (+ 644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49484: BlockingRecv error: recv got EOF from 127.0.0.1:49484 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260811 02:51:38.453532 28667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:38.451626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49490 (local address 127.21.236.189:45999)
0811 02:51:38.451752 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.451757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.452427 (+ 670us) server_negotiation.cc:407] Connection header received
0811 02:51:38.452667 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.452670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.452728 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.452780 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.453381 (+ 601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49490: BlockingRecv error: recv got EOF from 127.0.0.1:49490 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 02:51:38.478932 28667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:38.477636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49492 (local address 127.21.236.189:45999)
0811 02:51:38.477777 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.477781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.477793 (+ 12us) server_negotiation.cc:407] Connection header received
0811 02:51:38.477843 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.477845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.477888 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.477991 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.478807 (+ 816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49492: BlockingRecv error: recv got EOF from 127.0.0.1:49492 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 02:51:38.514057 28667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:38.512932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49494 (local address 127.21.236.189:45999)
0811 02:51:38.513084 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.513088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.513135 (+ 47us) server_negotiation.cc:407] Connection header received
0811 02:51:38.513300 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.513303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.513358 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.513460 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.513949 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49494: BlockingRecv error: recv got EOF from 127.0.0.1:49494 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 02:51:38.582052 28667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:38.580771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49508 (local address 127.21.236.189:45999)
0811 02:51:38.580949 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.580954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.581099 (+ 145us) server_negotiation.cc:407] Connection header received
0811 02:51:38.581217 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.581219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.581273 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.581378 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.581873 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49508: BlockingRecv error: recv got EOF from 127.0.0.1:49508 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 02:51:38.713812 28667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:38.712594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49516 (local address 127.21.236.189:45999)
0811 02:51:38.712759 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.712763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.712908 (+ 145us) server_negotiation.cc:407] Connection header received
0811 02:51:38.713033 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.713036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.713089 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.713179 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.713706 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49516: BlockingRecv error: recv got EOF from 127.0.0.1:49516 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 02:51:38.973713 28667 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:38.972238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49518 (local address 127.21.236.189:45999)
0811 02:51:38.972399 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:38.972407 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:38.972486 (+ 79us) server_negotiation.cc:407] Connection header received
0811 02:51:38.972870 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:38.972874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:38.972942 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:38.973052 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:38.973564 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49518: BlockingRecv error: recv got EOF from 127.0.0.1:49518 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 02:51:39.243645 28821 catalog_manager.cc:1259] Loaded cluster ID: 2e23cb8f9cb0415cb60e1f6136ed8314
I20260811 02:51:39.243748 28821 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a: loading cluster ID for follower catalog manager: success
I20260811 02:51:39.245611 28821 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a: acquiring CA information for follower catalog manager: success
I20260811 02:51:39.246857 28821 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c4c8d1dcc0e94b67b3a9510461cadc8a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:51:39.268041 28867 catalog_manager.cc:1259] Loaded cluster ID: 2e23cb8f9cb0415cb60e1f6136ed8314
I20260811 02:51:39.268141 28867 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33: loading cluster ID for follower catalog manager: success
I20260811 02:51:39.269925 28867 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33: acquiring CA information for follower catalog manager: success
I20260811 02:51:39.271229 28867 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c1bbd6cd02ff4a7fbee8e9a02bb1ae33: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:51:39.492091 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:39.488135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49522 (local address 127.21.236.189:45999)
0811 02:51:39.488414 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:39.488418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:39.488435 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:51:39.488539 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:39.488541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:39.488686 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:39.488786 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:39.489055 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:39.489631 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:39.490548 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:39.490776 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:39.490856 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:39.491164 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:39.491620 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:39.491713 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:39.491909 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 02:51:39.491964 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":84,"threads_started":1}
I20260811 02:51:39.499927 29009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:39.495795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51896 (local address 127.21.236.188:45765)
0811 02:51:39.496066 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:39.496070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:39.496095 (+ 25us) server_negotiation.cc:407] Connection header received
0811 02:51:39.496189 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:39.496192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:39.496311 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:39.496426 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:39.496697 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:39.497293 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:39.498188 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:39.498457 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:39.498556 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:39.498842 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:39.499419 (+ 577us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:39.499524 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:39.499624 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 02:51:39.499678 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":85,"threads_started":1}
I20260811 02:51:39.508114 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:39.503500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49524 (local address 127.21.236.189:45999)
0811 02:51:39.503650 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:39.503655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:39.503884 (+ 229us) server_negotiation.cc:407] Connection header received
0811 02:51:39.504142 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:39.504146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:39.504226 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:39.504284 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:39.504738 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:39.505649 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:39.506725 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:39.507071 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:39.507204 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:39.507514 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:39.507614 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:39.507668 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:39.507858 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 02:51:39.507934 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 02:51:39.508315 29009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:39.503938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51906 (local address 127.21.236.188:45765)
0811 02:51:39.504086 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:39.504090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:39.504502 (+ 412us) server_negotiation.cc:407] Connection header received
0811 02:51:39.504613 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:39.504617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:39.504684 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:39.504759 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:39.505013 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:39.505571 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:39.506932 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:39.507331 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:39.507459 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:39.507746 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:39.507857 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:39.507934 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:39.508109 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 02:51:39.508181 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 02:51:39.508965 29021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:39.503451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49200 (local address 127.21.236.190:33965)
0811 02:51:39.503819 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:39.503824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:39.503841 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:51:39.504013 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:39.504018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:39.504194 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:39.504348 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:39.504843 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:39.505713 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:39.506991 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:39.507412 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:39.507519 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:39.508010 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:39.508416 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:39.508516 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:39.508689 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 02:51:39.508757 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":113,"threads_started":1}
I20260811 02:51:39.512347 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 28379
I20260811 02:51:39.533777 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 28523
I20260811 02:51:39.546669 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 28669
2026-08-11T02:51:39Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3546 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 02:51:39 dist-test-slave-n326 krb5kdc[29028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 02:51:39 dist-test-slave-n326 krb5kdc[29028](info): set up 2 sockets
Aug 11 02:51:39 dist-test-slave-n326 krb5kdc[29028](info): commencing operation
krb5kdc: starting...
W20260811 02:51:40.890432 22450 mini_kdc.cc:121] Time spent starting KDC: real 1.317s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/test-user.keytab.
Aug 11 02:51:40 dist-test-slave-n326 krb5kdc[29028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T02:51:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T02:51:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.236.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
Entry for principal kudu/127.21.236.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab.
I20260811 02:51:41.035769 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:40665
--webserver_interface=127.21.236.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:40665,127.21.236.189:36581,127.21.236.188:41101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 02:51:41.152336 29049 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 02:51:41.152666 29049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:41.152690 29049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:41.152731 29049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:41.156781 29049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:41.156845 29049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:41.156865 29049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:41.156881 29049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:41.156896 29049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:41.162199 29049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:40665,127.21.236.189:36581,127.21.236.188:41101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.190.keytab
--principal=kudu/127.21.236.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.190:40665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.236.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:41.163496 29049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:41.164499 29049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:41.173671 29061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:41.173671 29060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:41.173740 29063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:41.173985 29049 server_base.cc:1034] running on GCE node
I20260811 02:51:41.174803 29049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:41.176120 29049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:41.177306 29049 hybrid_clock.cc:648] HybridClock initialized: now 1786416701177274 us; error 42 us; skew 500 ppm
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:41.183920 29049 init.cc:377] Logged in from keytab as kudu/127.21.236.190@KRBTEST.COM (short username kudu)
I20260811 02:51:41.185045 29049 webserver.cc:460] Webserver started at http://127.21.236.190:35853/ using document root <none> and password file <none>
I20260811 02:51:41.185652 29049 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:41.185719 29049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:41.185911 29049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:41.188299 29049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/data/instance:
uuid: "8f5b9baf4bba432ca78348425536edab"
format_stamp: "Formatted at 2026-08-11 02:51:41 on dist-test-slave-n326"
server_key: "57c009e7b72efe40625cfd8f4f53976e"
server_key_iv: "f1d31c0acc4e2d23d6f151c389fccdac"
server_key_version: "encryptionkey@0"
I20260811 02:51:41.188877 29049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/wal/instance:
uuid: "8f5b9baf4bba432ca78348425536edab"
format_stamp: "Formatted at 2026-08-11 02:51:41 on dist-test-slave-n326"
server_key: "57c009e7b72efe40625cfd8f4f53976e"
server_key_iv: "f1d31c0acc4e2d23d6f151c389fccdac"
server_key_version: "encryptionkey@0"
I20260811 02:51:41.193145 29049 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 02:51:41.195905 29076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:41.197261 29049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 02:51:41.197377 29049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/wal
uuid: "8f5b9baf4bba432ca78348425536edab"
format_stamp: "Formatted at 2026-08-11 02:51:41 on dist-test-slave-n326"
server_key: "57c009e7b72efe40625cfd8f4f53976e"
server_key_iv: "f1d31c0acc4e2d23d6f151c389fccdac"
server_key_version: "encryptionkey@0"
I20260811 02:51:41.197471 29049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:41.210834 29049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:41.214123 29049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:41.214294 29049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:41.221936 29049 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.190:40665
I20260811 02:51:41.222288 29160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.190:40665 every 8 connection(s)
I20260811 02:51:41.222999 29049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/data/info.pb
I20260811 02:51:41.224077 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 29049
I20260811 02:51:41.224242 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-0/wal/instance
I20260811 02:51:41.224568 22450 external_mini_cluster.cc:1428] Setting key 7dea23cd9d04d46a4876d7a56579bd44
I20260811 02:51:41.226087 29161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:41.230052 29161 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.236.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.189@KRBTEST.COM" created.
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:51:41.247328 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.232822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40665 (local address 127.0.0.1:54462)
0811 02:51:41.233582 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.233649 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:41.234775 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:41.235544 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:41.235549 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:41.235992 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:41.236239 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.236253 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.237484 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.237487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.238217 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.238226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.238977 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.238982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.239195 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.239838 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:41.239855 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:41.241404 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:41.244026 (+ 2622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.244035 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.244044 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.244330 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.244807 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.244810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.244812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.244967 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.245809 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:41.245814 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:41.245950 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 02:51:41.246109 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 02:51:41.246429 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":5,"thread_start_us":230,"threads_started":1}
I20260811 02:51:41.247344 29168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.233994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54462 (local address 127.21.236.190:40665)
0811 02:51:41.234452 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.234462 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.234526 (+ 64us) server_negotiation.cc:407] Connection header received
0811 02:51:41.235146 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.235150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.235346 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.235545 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.236398 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.237343 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.238354 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.238760 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.238948 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.241589 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.241597 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.241613 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.241644 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.243887 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.244502 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.244509 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.244511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.244678 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.245082 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.245086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.245088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.245659 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.245830 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.246228 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.246493 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":164,"server-negotiator.queue_time_us":353,"thread_start_us":228,"threads_started":1}
I20260811 02:51:41.249975 29161 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:51:41.250907 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.250483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:36581 (local address 127.0.0.1:57340)
0811 02:51:41.250650 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.250785 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:36581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260811 02:51:41.251183 29081 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.236.189:36581: connect: Connection refused (error 111)
I20260811 02:51:41.252267 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.251986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:36581 (local address 127.0.0.1:57348)
0811 02:51:41.252160 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.252215 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:36581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260811 02:51:41.252440 29161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.189:36581: Network error: Client connection negotiation failed: client connection to 127.21.236.189:36581: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.189.keytab.
Entry for principal kudu/127.21.236.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.189.keytab.
I20260811 02:51:41.258826 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.189:36581
--webserver_interface=127.21.236.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.189.keytab
--principal=kudu/127.21.236.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:40665,127.21.236.189:36581,127.21.236.188:41101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:51:41.301851 29161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } attempt: 1
I20260811 02:51:41.302867 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.302501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:36581 (local address 127.0.0.1:57350)
0811 02:51:41.302709 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.302779 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:36581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 02:51:41.303838 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.303560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:36581 (local address 127.0.0.1:57354)
0811 02:51:41.303707 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.303769 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:36581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 02:51:41.303989 29161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.189:36581: Network error: Client connection negotiation failed: client connection to 127.21.236.189:36581: connect: Connection refused (error 111)
I20260811 02:51:41.372202 29161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } attempt: 2
I20260811 02:51:41.373174 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.372840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:36581 (local address 127.0.0.1:57362)
0811 02:51:41.373034 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.373096 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:36581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 02:51:41.374145 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.373780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:36581 (local address 127.0.0.1:57364)
0811 02:51:41.373949 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.374075 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.189:36581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 02:51:41.374284 29161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.189:36581: Network error: Client connection negotiation failed: client connection to 127.21.236.189:36581: connect: Connection refused (error 111)
W20260811 02:51:41.381037 29176 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 02:51:41.381385 29176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:41.381462 29176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:41.381554 29176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:41.385739 29176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:41.385824 29176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:41.385866 29176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:41.385898 29176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:41.385950 29176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:41.391584 29176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:40665,127.21.236.189:36581,127.21.236.188:41101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.189.keytab
--principal=kudu/127.21.236.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.189:36581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.236.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:41.392787 29176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:41.393869 29176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:41.403393 29185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:41.403445 29187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:41.403393 29182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:41.403797 29176 server_base.cc:1034] running on GCE node
I20260811 02:51:41.404343 29176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:41.405566 29176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:41.406842 29176 hybrid_clock.cc:648] HybridClock initialized: now 1786416701406821 us; error 34 us; skew 500 ppm
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:41.410117 29176 init.cc:377] Logged in from keytab as kudu/127.21.236.189@KRBTEST.COM (short username kudu)
I20260811 02:51:41.411237 29176 webserver.cc:460] Webserver started at http://127.21.236.189:39997/ using document root <none> and password file <none>
I20260811 02:51:41.411863 29176 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:41.411970 29176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:41.412225 29176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:41.414429 29176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/data/instance:
uuid: "c6891dee07af42b481cb226c1c25548d"
format_stamp: "Formatted at 2026-08-11 02:51:41 on dist-test-slave-n326"
server_key: "dc04fdb79adf21c58535c4aabd367dcf"
server_key_iv: "84da0efb35b1948fc0758be886a3ed50"
server_key_version: "encryptionkey@0"
I20260811 02:51:41.415040 29176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/wal/instance:
uuid: "c6891dee07af42b481cb226c1c25548d"
format_stamp: "Formatted at 2026-08-11 02:51:41 on dist-test-slave-n326"
server_key: "dc04fdb79adf21c58535c4aabd367dcf"
server_key_iv: "84da0efb35b1948fc0758be886a3ed50"
server_key_version: "encryptionkey@0"
I20260811 02:51:41.419607 29176 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 02:51:41.422240 29199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:41.423694 29176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:41.423849 29176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/wal
uuid: "c6891dee07af42b481cb226c1c25548d"
format_stamp: "Formatted at 2026-08-11 02:51:41 on dist-test-slave-n326"
server_key: "dc04fdb79adf21c58535c4aabd367dcf"
server_key_iv: "84da0efb35b1948fc0758be886a3ed50"
server_key_version: "encryptionkey@0"
I20260811 02:51:41.423979 29176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:41.450796 29176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:41.454309 29176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:41.454555 29176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:41.462378 29176 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.189:36581
I20260811 02:51:41.462677 29301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.189:36581 every 8 connection(s)
I20260811 02:51:41.463531 29176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/data/info.pb
I20260811 02:51:41.465481 29161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } attempt: 3
I20260811 02:51:41.466845 29302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:41.467887 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 29176
I20260811 02:51:41.468035 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-1/wal/instance
I20260811 02:51:41.468379 22450 external_mini_cluster.cc:1428] Setting key f62ed79db0f50befaf1fee80971c57e5
I20260811 02:51:41.471704 29302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for kudu/127.21.236.189@KRBTEST.COM
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.189@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.236.188@KRBTEST.COM; defaulting to no policy
I20260811 02:51:41.481350 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.466103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:36581 (local address 127.0.0.1:57376)
0811 02:51:41.466268 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.466289 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:41.466403 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:41.469387 (+ 2984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:41.469390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:41.469626 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:41.469786 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.469796 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.471695 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.471700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.472529 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.472544 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.473454 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.473458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.473575 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.474178 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:41.474192 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:41.475958 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:41.478822 (+ 2864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.478825 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.478827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.479137 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.479524 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.479534 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.479536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.479660 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.480652 (+ 992us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:41.480658 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:41.480796 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 02:51:41.480963 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 02:51:41.481140 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 02:51:41.482966 29304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.466954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57376 (local address 127.21.236.189:36581)
0811 02:51:41.467450 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.467466 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.467512 (+ 46us) server_negotiation.cc:407] Connection header received
0811 02:51:41.468546 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.468570 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.469026 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.469415 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.469932 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.471539 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.472694 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.473298 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.473483 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.476220 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.476229 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.476245 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.476282 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.478681 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.479260 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.479272 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.479277 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.479400 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.479782 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.479789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.479791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.480427 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.480587 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.481755 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.482083 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":133,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.236.188@KRBTEST.COM" created.
I20260811 02:51:41.485428 29161 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:51:41.485842 29168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.473900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54468 (local address 127.21.236.190:40665)
0811 02:51:41.474059 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.474063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.474831 (+ 768us) server_negotiation.cc:407] Connection header received
0811 02:51:41.475108 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.475113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.475181 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.475281 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.476311 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.477205 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.478173 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.479016 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.479150 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.481502 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.481507 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.481509 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.481543 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.483203 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.483683 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.483687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.483689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.483787 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.484408 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.484411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.484412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.484615 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.484696 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.485516 (+ 820us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.485690 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 02:51:41.486222 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.486000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36524)
0811 02:51:41.486100 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.486156 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 02:51:41.487054 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.486796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36532)
0811 02:51:41.486946 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.486996 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 02:51:41.487314 29161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:41101: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
I20260811 02:51:41.487448 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.474212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40665 (local address 127.0.0.1:54468)
0811 02:51:41.474677 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.474742 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:41.474938 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:41.475381 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:41.475388 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:41.475929 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:41.476174 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.476184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.477372 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.477377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.478025 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.478043 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.479188 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.479194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.479355 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.479975 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:41.479987 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:41.481346 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:41.483333 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.483340 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.483345 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.483573 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.484064 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.484071 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.484077 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.484280 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.484801 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:41.484807 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:41.484942 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 02:51:41.485123 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 02:51:41.485351 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":114,"threads_started":1}
I20260811 02:51:41.489109 29302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.189@KRBTEST.COM for kudu/127.21.236.189@KRBTEST.COM
I20260811 02:51:41.499333 29304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.489771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57378 (local address 127.21.236.189:36581)
0811 02:51:41.489868 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.489872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.489887 (+ 15us) server_negotiation.cc:407] Connection header received
0811 02:51:41.489967 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.489972 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.490011 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.490119 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.490537 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.491204 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.492429 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.492925 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.493070 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.495625 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.495634 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.495639 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.495673 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.497327 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.497854 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.497857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.497859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.497938 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.498288 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.498294 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.498298 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.498496 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.498611 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.498842 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.499173 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":33}
I20260811 02:51:41.499349 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.489673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:36581 (local address 127.0.0.1:57378)
0811 02:51:41.489773 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.489789 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:41.489874 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:41.490131 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:41.490134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:41.490325 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:41.490416 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.490421 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.491352 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.491358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.492284 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.492295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.493114 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.493117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.493199 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.493835 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:41.493847 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:41.495450 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:41.497531 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.497534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.497536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.497758 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.498066 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.498069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.498071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.498156 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.498611 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:41.498613 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:41.498701 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 02:51:41.498795 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 02:51:41.498982 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 02:51:41.500445 29302 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:51:41.501794 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.501322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36546)
0811 02:51:41.501612 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.501699 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
W20260811 02:51:41.502110 29201 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
I20260811 02:51:41.503300 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.503032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36554)
0811 02:51:41.503180 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.503233 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260811 02:51:41.503489 29302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:41101: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.236.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.188.keytab.
Entry for principal kudu/127.21.236.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.188.keytab.
I20260811 02:51:41.508267 22450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
/tmp/dist-test-taskQxaM5w/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.188:41101
--webserver_interface=127.21.236.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.188.keytab
--principal=kudu/127.21.236.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.236.148:39231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.236.190:40665,127.21.236.189:36581,127.21.236.188:41101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 02:51:41.518528 29161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } attempt: 1
I20260811 02:51:41.519717 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.519274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36562)
0811 02:51:41.519498 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.519590 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 02:51:41.520685 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.520324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36578)
0811 02:51:41.520480 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.520586 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 02:51:41.520896 29161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:41101: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
I20260811 02:51:41.552834 29302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } attempt: 1
I20260811 02:51:41.553957 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.553566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36590)
0811 02:51:41.553790 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.553853 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260811 02:51:41.555181 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.554711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36602)
0811 02:51:41.554864 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.555028 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 02:51:41.555342 29302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:41101: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
I20260811 02:51:41.596124 29161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } attempt: 2
I20260811 02:51:41.597162 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.596768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36616)
0811 02:51:41.596982 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.597063 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 02:51:41.598101 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.597829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36624)
0811 02:51:41.597969 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.598026 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 02:51:41.598281 29161 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:41101: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
I20260811 02:51:41.623621 29302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } attempt: 2
I20260811 02:51:41.624512 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.624206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36636)
0811 02:51:41.624355 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.624423 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 02:51:41.625344 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.625128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36648)
0811 02:51:41.625235 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.625289 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 02:51:41.625523 29302 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.236.188:41101: Network error: Client connection negotiation failed: client connection to 127.21.236.188:41101: connect: Connection refused (error 111)
W20260811 02:51:41.627832 29309 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 02:51:41.628118 29309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 02:51:41.628145 29309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 02:51:41.628234 29309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 02:51:41.632295 29309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 02:51:41.632351 29309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 02:51:41.632371 29309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 02:51:41.632388 29309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 02:51:41.632404 29309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 02:51:41.637702 29309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.236.148:39231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.236.190:40665,127.21.236.189:36581,127.21.236.188:41101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/krb5kdc/kudu_127.21.236.188.keytab
--principal=kudu/127.21.236.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.236.188:41101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.236.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 02:09:49 UTC on ccycloud.quasar-fsnqqc.root.comops.site
build id 4579570
I20260811 02:51:41.638757 29309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 02:51:41.639760 29309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 02:51:41.648557 29328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:41.648546 29329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 02:51:41.648546 29332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 02:51:41.648978 29309 server_base.cc:1034] running on GCE node
I20260811 02:51:41.649595 29309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 02:51:41.650754 29309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 02:51:41.651940 29309 hybrid_clock.cc:648] HybridClock initialized: now 1786416701651913 us; error 34 us; skew 500 ppm
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 02:51:41.655041 29309 init.cc:377] Logged in from keytab as kudu/127.21.236.188@KRBTEST.COM (short username kudu)
I20260811 02:51:41.656167 29309 webserver.cc:460] Webserver started at http://127.21.236.188:44983/ using document root <none> and password file <none>
I20260811 02:51:41.656737 29309 fs_manager.cc:359] Metadata directory not provided
I20260811 02:51:41.656800 29309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 02:51:41.656985 29309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 02:51:41.659235 29309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/data/instance:
uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc"
format_stamp: "Formatted at 2026-08-11 02:51:41 on dist-test-slave-n326"
server_key: "23c998635589889e315ce54b8919cb8d"
server_key_iv: "e8977e09157bcce3ecfd62c8cf1959d2"
server_key_version: "encryptionkey@0"
I20260811 02:51:41.659819 29309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/wal/instance:
uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc"
format_stamp: "Formatted at 2026-08-11 02:51:41 on dist-test-slave-n326"
server_key: "23c998635589889e315ce54b8919cb8d"
server_key_iv: "e8977e09157bcce3ecfd62c8cf1959d2"
server_key_version: "encryptionkey@0"
I20260811 02:51:41.664008 29309 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 02:51:41.666908 29341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 02:51:41.668372 29309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 02:51:41.668500 29309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/data,/tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/wal
uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc"
format_stamp: "Formatted at 2026-08-11 02:51:41 on dist-test-slave-n326"
server_key: "23c998635589889e315ce54b8919cb8d"
server_key_iv: "e8977e09157bcce3ecfd62c8cf1959d2"
server_key_version: "encryptionkey@0"
I20260811 02:51:41.668602 29309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 02:51:41.684098 29309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 02:51:41.687402 29309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 02:51:41.687577 29309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 02:51:41.695399 29309 rpc_server.cc:307] RPC server started. Bound to: 127.21.236.188:41101
I20260811 02:51:41.695780 29436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.236.188:41101 every 8 connection(s)
I20260811 02:51:41.696511 29309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/data/info.pb
I20260811 02:51:41.697454 29161 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } attempt: 3
I20260811 02:51:41.700069 29437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 02:51:41.704881 29437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:51:41.706543 22450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu as pid 29309
I20260811 02:51:41.706697 22450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQxaM5w/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786416395682572-22450-0/minicluster-data/master-2/wal/instance
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.190@KRBTEST.COM for kudu/127.21.236.188@KRBTEST.COM
I20260811 02:51:41.707050 22450 external_mini_cluster.cc:1428] Setting key 09e3b2497fa3a2b41b76cf61a333e1a7
I20260811 02:51:41.712671 29167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.698080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36664)
0811 02:51:41.698253 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.698273 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:41.698389 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:41.700947 (+ 2558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:41.700950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:41.701210 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:41.701331 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.701340 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.702892 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.702898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.703919 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.703936 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.704927 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.704932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.705024 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.705653 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:41.705671 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:41.707360 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:41.710337 (+ 2977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.710340 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.710343 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.710584 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.711004 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.711009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.711013 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.711150 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.712066 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:41.712081 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:41.712197 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 02:51:41.712308 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 02:51:41.712474 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.188@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
I20260811 02:51:41.714553 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.698910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36664 (local address 127.21.236.188:41101)
0811 02:51:41.699433 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.699449 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.699488 (+ 39us) server_negotiation.cc:407] Connection header received
0811 02:51:41.700267 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.700287 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.700645 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.701010 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.701524 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.702743 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.704077 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.704778 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.704973 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.707608 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.707615 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.707633 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.707672 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.710207 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.710722 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.710728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.710734 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.710894 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.711284 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.711289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.711292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.711921 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.712905 (+ 984us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.713132 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.713402 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":172,"threads_started":1}
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.188@KRBTEST.COM
I20260811 02:51:41.716714 29302 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } attempt: 3
I20260811 02:51:41.718263 29168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.706871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54470 (local address 127.21.236.190:40665)
0811 02:51:41.707017 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.707021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.707474 (+ 453us) server_negotiation.cc:407] Connection header received
0811 02:51:41.707700 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.707704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.707757 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.707809 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.708760 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.709392 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.710377 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.710724 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.710830 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.713705 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.713709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.713712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.713739 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.715586 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.716181 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.716183 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.716185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.716261 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.716563 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.716567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.716569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.716791 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.716912 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.717680 (+ 768us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.718065 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 02:51:41.719113 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.706809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.190:40665 (local address 127.0.0.1:54470)
0811 02:51:41.707329 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.707388 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:41.707576 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:41.708000 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:41.708006 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:41.708501 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:41.708627 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.708633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.709529 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.709532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.710239 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.710248 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.710845 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.710849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.710996 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.711635 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:41.711650 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:41.713529 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:41.715721 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.715733 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.715740 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.716062 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.716365 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.716369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.716371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.716463 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.716917 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:41.716927 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:41.717061 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 02:51:41.718663 (+ 1602us) client_negotiation.cc:240] Negotiation successful
0811 02:51:41.718861 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":153,"threads_started":1}
I20260811 02:51:41.720453 29437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:51:41.720494 29161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab: Bootstrap starting.
I20260811 02:51:41.721315 29444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.708198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.21.236.188:41101)
0811 02:51:41.708899 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.708923 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.708936 (+ 13us) server_negotiation.cc:407] Connection header received
0811 02:51:41.708968 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.708971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.709010 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.709094 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.710106 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.710934 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.712049 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.712272 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.712391 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.716137 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.716140 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.716143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.716169 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.718760 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.719445 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.719450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.719453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.719595 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.719930 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.719934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.719936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.720137 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.720309 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.720996 (+ 687us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.721134 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":108,"threads_started":1}
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.189@KRBTEST.COM for kudu/127.21.236.188@KRBTEST.COM
I20260811 02:51:41.723667 29161 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:41.724740 29161 log.cc:824] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:41.727181 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.717252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36684)
0811 02:51:41.717505 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.717522 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:41.717607 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:41.717928 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:41.717931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:41.718161 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:41.718295 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.718304 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.719512 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.719520 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.720430 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.720443 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.721101 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.721105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.721195 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.721663 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:41.721680 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:41.723359 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:41.725432 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.725436 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.725438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.725724 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.726057 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.726060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.726062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.726156 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.726626 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:41.726630 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:41.726725 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 02:51:41.726810 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 02:51:41.726951 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 02:51:41.727387 29161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab: No bootstrap required, opened a new log
I20260811 02:51:41.727774 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.717537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36684 (local address 127.21.236.188:41101)
0811 02:51:41.717660 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.717666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.717682 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:51:41.717724 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.717730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.717783 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.718424 (+ 641us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.718527 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.719377 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.720593 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.720941 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.721452 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.723513 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.723516 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.723522 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.723547 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.725285 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.725860 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.725864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.725866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.725952 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.726269 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.726272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.726274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.726499 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.727384 (+ 885us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.727474 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.727617 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.188@KRBTEST.COM for kudu/127.21.236.189@KRBTEST.COM
I20260811 02:51:41.731786 29302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d: Bootstrap starting.
I20260811 02:51:41.732131 29161 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } }
I20260811 02:51:41.732367 29161 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f5b9baf4bba432ca78348425536edab, State: Initialized, Role: FOLLOWER
I20260811 02:51:41.732719 29161 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } }
I20260811 02:51:41.733067 29304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.721351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57380 (local address 127.21.236.189:36581)
0811 02:51:41.721747 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.721751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.721980 (+ 229us) server_negotiation.cc:407] Connection header received
0811 02:51:41.722148 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.722151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.722193 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.722242 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.723053 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.723765 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.725400 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.725734 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.725815 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.729255 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.729262 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.729267 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.729297 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.731058 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.731638 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.731642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.731643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.731732 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.732084 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.732087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.732089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.732311 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.732387 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.732781 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.732903 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326}
I20260811 02:51:41.733497 29447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } } }
I20260811 02:51:41.733639 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.721027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.189:36581 (local address 127.0.0.1:57380)
0811 02:51:41.721885 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.721903 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:41.722044 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:41.722374 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:41.722377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:41.722538 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:41.722901 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.722924 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.724198 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.724201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.725251 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.725264 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.725974 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.725977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.726076 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.726523 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:41.726537 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:41.729101 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:41.731224 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.731228 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.731230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.731492 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.731863 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.731867 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.731869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.731967 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.732470 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:41.732473 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:41.732568 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 02:51:41.733282 (+ 714us) client_negotiation.cc:240] Negotiation successful
0811 02:51:41.733457 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"thread_start_us":101,"threads_started":1}
I20260811 02:51:41.734025 29447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:51:41.734361 29161 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:41.734609 29437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } has no permanent_uuid. Determining permanent_uuid...
I20260811 02:51:41.734913 29302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:41.735954 29302 log.cc:824] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d: Log is configured to *not* fsync() on all Append() calls
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.190@KRBTEST.COM
W20260811 02:51:41.740792 29466 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:51:41.740938 29466 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416701, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.236.188@KRBTEST.COM for kudu/127.21.236.188@KRBTEST.COM
I20260811 02:51:41.740125 29049 master_runner.cc:186] Error getting master registration for 127.21.236.189:36581: OK, instance_id { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" instance_seqno: 1786416701460154 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 02:51:41.744320 29049 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:51:41.744872 29302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d: No bootstrap required, opened a new log
I20260811 02:51:41.746083 29442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.735268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.236.188:41101 (local address 127.0.0.1:36686)
0811 02:51:41.735559 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 02:51:41.735586 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 02:51:41.735669 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 02:51:41.736006 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 02:51:41.736009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 02:51:41.736192 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 02:51:41.736285 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.736293 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.737383 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.737386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.738246 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 02:51:41.738257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.738923 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.738930 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 02:51:41.739016 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.739948 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 02:51:41.739966 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 02:51:41.741679 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 02:51:41.744152 (+ 2473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.744157 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.744159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.744452 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.744806 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 02:51:41.744809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 02:51:41.744811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 02:51:41.744926 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 02:51:41.745411 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 02:51:41.745415 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 02:51:41.745528 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 02:51:41.745659 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 02:51:41.745839 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245}
I20260811 02:51:41.746106 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.735398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36686 (local address 127.21.236.188:41101)
0811 02:51:41.735498 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.735502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.735752 (+ 250us) server_negotiation.cc:407] Connection header received
0811 02:51:41.735782 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.735786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.735838 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.735905 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.736427 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.737256 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.738379 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.738698 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.738807 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.741868 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.741873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.741876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.741905 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.743996 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.744572 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.744578 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.744581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.744682 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.745043 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.745045 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.745047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.745283 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.745399 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.745740 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.745915 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":35}
I20260811 02:51:41.749444 29302 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } }
I20260811 02:51:41.750064 29437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc: Bootstrap starting.
I20260811 02:51:41.750908 29302 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6891dee07af42b481cb226c1c25548d, State: Initialized, Role: FOLLOWER
I20260811 02:51:41.753274 29437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc: Neither blocks nor log segments found. Creating new log.
I20260811 02:51:41.753533 29302 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } }
I20260811 02:51:41.754309 29437 log.cc:824] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc: Log is configured to *not* fsync() on all Append() calls
I20260811 02:51:41.754909 29168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.722744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54474 (local address 127.21.236.190:40665)
0811 02:51:41.722843 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.722846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.723035 (+ 189us) server_negotiation.cc:407] Connection header received
0811 02:51:41.723270 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.723274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.723325 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.723380 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.724426 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.725290 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.726504 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.726859 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.726940 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.750480 (+ 23540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.750485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.750488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.750521 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.752545 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.753157 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.753165 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.753167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.753266 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.753659 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.753663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.753665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.753909 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.754116 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.754562 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.754700 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 02:51:41.755204 29302 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:41.756283 29470 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } } }
I20260811 02:51:41.756494 29470 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:51:41.757684 29437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc: No bootstrap required, opened a new log
I20260811 02:51:41.762428 29437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } }
I20260811 02:51:41.762764 29437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e923cfc211e4b6ca01a2a469cb9e8dc, State: Initialized, Role: FOLLOWER
I20260811 02:51:41.763201 29437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } }
Aug 11 02:51:41 dist-test-slave-n326 krb5kdc[29028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786416700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.236.189@KRBTEST.COM
I20260811 02:51:41.765036 29437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 02:51:41.763443 29176 master_runner.cc:186] Error getting master registration for 127.21.236.188:41101: OK, instance_id { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" instance_seqno: 1786416701693094 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 02:51:41.770434 29304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.757368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57388 (local address 127.21.236.189:36581)
0811 02:51:41.757561 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.757567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.757587 (+ 20us) server_negotiation.cc:407] Connection header received
0811 02:51:41.757634 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.757638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.757695 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.757780 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.758736 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.759636 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.760878 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.761252 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.761352 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.764856 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.764860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.764863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.764892 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.766913 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.767497 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.767502 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.767504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.768579 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.768918 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.768922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.768924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.769164 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.769371 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.770112 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.770265 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 02:51:41.771914 29505 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:51:41.772004 29505 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:51:41.772794 29176 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:51:41.773996 22450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 02:51:41.774003 29309 master_runner.cc:263] No leader master found from master 6e923cfc211e4b6ca01a2a469cb9e8dc
I20260811 02:51:41.774061 29309 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 02:51:41.775355 29115 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8f5b9baf4bba432ca78348425536edab"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54474
I20260811 02:51:41.775620 29115 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 02:51:41.775743 29115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [term 0 FOLLOWER]: Advancing to term 1
W20260811 02:51:41.776569 29489 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 02:51:41.776638 29489 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 02:51:41.777303 29115 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } }
I20260811 02:51:41.778745 29251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f5b9baf4bba432ca78348425536edab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c6891dee07af42b481cb226c1c25548d"
I20260811 02:51:41.778888 29115 leader_election.cc:288] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [CANDIDATE]: Term 1 election: Requested vote from peers c6891dee07af42b481cb226c1c25548d (127.21.236.189:36581), 6e923cfc211e4b6ca01a2a469cb9e8dc (127.21.236.188:41101)
I20260811 02:51:41.779575 29387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f5b9baf4bba432ca78348425536edab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc"
I20260811 02:51:41.779804 29387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:41.781255 29387 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f5b9baf4bba432ca78348425536edab in term 1.
I20260811 02:51:41.778991 29251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d [term 0 FOLLOWER]: Advancing to term 1
I20260811 02:51:41.781751 29079 leader_election.cc:302] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e923cfc211e4b6ca01a2a469cb9e8dc, 8f5b9baf4bba432ca78348425536edab; no voters:
I20260811 02:51:41.782058 29447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [term 1 FOLLOWER]: Leader election won for term 1
I20260811 02:51:41.782310 29447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [term 1 LEADER]: Becoming Leader. State: Replica: 8f5b9baf4bba432ca78348425536edab, State: Running, Role: LEADER
I20260811 02:51:41.782678 29447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } }
I20260811 02:51:41.782936 29251 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f5b9baf4bba432ca78348425536edab in term 1.
I20260811 02:51:41.787424 29490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } } }
I20260811 02:51:41.787717 29490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:51:41.787706 29527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f5b9baf4bba432ca78348425536edab. Latest consensus state: current_term: 1 leader_uuid: "8f5b9baf4bba432ca78348425536edab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } } }
I20260811 02:51:41.787845 29527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:41.790016 29530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 02:51:41.791718 29530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 02:51:41.800338 29387 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc [term 1 FOLLOWER]: Refusing update from remote peer 8f5b9baf4bba432ca78348425536edab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:41.801240 29527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:41.806128 29490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f5b9baf4bba432ca78348425536edab. Latest consensus state: current_term: 1 leader_uuid: "8f5b9baf4bba432ca78348425536edab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } } }
I20260811 02:51:41.806286 29490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:51:41.807655 29251 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d [term 1 FOLLOWER]: Refusing update from remote peer 8f5b9baf4bba432ca78348425536edab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 02:51:41.808419 29527 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 02:51:41.809937 29440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.788399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36692 (local address 127.21.236.188:41101)
0811 02:51:41.788534 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.788539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.795617 (+ 7078us) server_negotiation.cc:407] Connection header received
0811 02:51:41.795677 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.795681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.795736 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.795814 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.797619 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.798417 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.799680 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.799991 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.800097 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.802817 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.802820 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.802823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.802855 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.806043 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.806619 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.806623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.806625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.806724 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.807477 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.807481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.807483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.807745 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.807897 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.809636 (+ 1739us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.809768 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:51:41.813197 29470 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f5b9baf4bba432ca78348425536edab. Latest consensus state: current_term: 1 leader_uuid: "8f5b9baf4bba432ca78348425536edab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } } }
I20260811 02:51:41.813350 29470 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:51:41.814803 29304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.789378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57396 (local address 127.21.236.189:36581)
0811 02:51:41.789551 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.789556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.795620 (+ 6064us) server_negotiation.cc:407] Connection header received
0811 02:51:41.795678 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.795681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.795736 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.795814 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.800661 (+ 4847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.801444 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.804681 (+ 3237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.805057 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.805148 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.810163 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.810168 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.810170 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.810201 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.812357 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.812922 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.812928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.812933 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.813036 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.813377 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.813381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.813382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.813647 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.813780 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.814459 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.814616 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 02:51:41.823904 29527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8f5b9baf4bba432ca78348425536edab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } } }
I20260811 02:51:41.824052 29527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:41.825059 29470 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8f5b9baf4bba432ca78348425536edab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } } }
I20260811 02:51:41.825199 29470 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:51:41.825695 29534 mvcc.cc:205] Tried to move back new op lower bound from 7317162810565222400 to 7317162810505834496. Current Snapshot: MvccSnapshot[applied={T|T < 7317162810565222400}]
I20260811 02:51:41.826772 29447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8f5b9baf4bba432ca78348425536edab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } } }
I20260811 02:51:41.826915 29447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab [sys.catalog]: This master's current role is: LEADER
I20260811 02:51:41.828399 29490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8f5b9baf4bba432ca78348425536edab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f5b9baf4bba432ca78348425536edab" member_type: VOTER last_known_addr { host: "127.21.236.190" port: 40665 } } peers { permanent_uuid: "c6891dee07af42b481cb226c1c25548d" member_type: VOTER last_known_addr { host: "127.21.236.189" port: 36581 } } peers { permanent_uuid: "6e923cfc211e4b6ca01a2a469cb9e8dc" member_type: VOTER last_known_addr { host: "127.21.236.188" port: 41101 } } }
I20260811 02:51:41.828533 29490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 02:51:41.829500 29544 mvcc.cc:205] Tried to move back new op lower bound from 7317162810565222400 to 7317162810505834496. Current Snapshot: MvccSnapshot[applied={T|T < 7317162810565222400}]
I20260811 02:51:41.829780 29536 mvcc.cc:205] Tried to move back new op lower bound from 7317162810565222400 to 7317162810505834496. Current Snapshot: MvccSnapshot[applied={T|T < 7317162810565222400}]
I20260811 02:51:41.834203 29168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:41.788938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54480 (local address 127.21.236.190:40665)
0811 02:51:41.789052 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.789057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.810722 (+ 21665us) server_negotiation.cc:407] Connection header received
0811 02:51:41.811368 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.811372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.811429 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.811488 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.815918 (+ 4430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.816733 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.827188 (+ 10455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:41.827551 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:41.827655 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:41.829903 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 02:51:41.829907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 02:51:41.829911 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 02:51:41.829941 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 02:51:41.831795 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.832337 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.832341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.832343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.832442 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 02:51:41.832811 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 02:51:41.832815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 02:51:41.832817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 02:51:41.833062 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 02:51:41.833213 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:41.833908 (+ 695us) server_negotiation.cc:299] Negotiation successful
0811 02:51:41.834043 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 02:51:41.836323 29530 catalog_manager.cc:1347] Generated new cluster ID: 79628256fff5443a9a62579f313eef56
I20260811 02:51:41.836390 29530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 02:51:41.853592 29530 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 02:51:41.855836 29530 catalog_manager.cc:1504] Loading token signing keys...
I20260811 02:51:41.869810 29530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f5b9baf4bba432ca78348425536edab: Generated new TSK 0
I20260811 02:51:41.870903 29530 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 02:51:41.909790 29168 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:41.905722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54494 (local address 127.21.236.190:40665)
0811 02:51:41.905869 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.905874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.908166 (+ 2292us) server_negotiation.cc:407] Connection header received
0811 02:51:41.908704 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.908708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.908777 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.908843 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.909636 (+ 793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54494: BlockingRecv error: recv got EOF from 127.0.0.1:54494 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 02:51:41.911329 29304 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:41.906603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.21.236.189:36581)
0811 02:51:41.906735 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.906739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.908599 (+ 1860us) server_negotiation.cc:407] Connection header received
0811 02:51:41.910409 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.910413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.910487 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.910548 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.911191 (+ 643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57398: BlockingRecv error: recv got EOF from 127.0.0.1:57398 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 02:51:41.913247 29440 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:41.907492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.21.236.188:41101)
0811 02:51:41.907645 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.907650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.912154 (+ 4504us) server_negotiation.cc:407] Connection header received
0811 02:51:41.912402 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.912405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.912477 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.912535 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.913104 (+ 569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36698: BlockingRecv error: recv got EOF from 127.0.0.1:36698 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 02:51:41.918627 29304 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:41.916780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57402 (local address 127.21.236.189:36581)
0811 02:51:41.916900 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.916904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.917548 (+ 644us) server_negotiation.cc:407] Connection header received
0811 02:51:41.917793 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.917797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.917895 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.917953 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.918519 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57402: BlockingRecv error: recv got EOF from 127.0.0.1:57402 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 02:51:41.924090 29304 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:41.922320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57416 (local address 127.21.236.189:36581)
0811 02:51:41.922428 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.922432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.923019 (+ 587us) server_negotiation.cc:407] Connection header received
0811 02:51:41.923268 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.923272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.923327 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.923385 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.923957 (+ 572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57416: BlockingRecv error: recv got EOF from 127.0.0.1:57416 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 02:51:41.932024 29304 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:41.929879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57430 (local address 127.21.236.189:36581)
0811 02:51:41.929991 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.929996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.930623 (+ 627us) server_negotiation.cc:407] Connection header received
0811 02:51:41.930842 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.930846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.930903 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.930962 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.931909 (+ 947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57430: BlockingRecv error: recv got EOF from 127.0.0.1:57430 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 02:51:41.940335 29304 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:41.938494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57434 (local address 127.21.236.189:36581)
0811 02:51:41.938651 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.938655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.939306 (+ 651us) server_negotiation.cc:407] Connection header received
0811 02:51:41.939536 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.939539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.939596 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.939654 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.940225 (+ 571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57434: BlockingRecv error: recv got EOF from 127.0.0.1:57434 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 02:51:41.952617 29304 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:41.950833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57436 (local address 127.21.236.189:36581)
0811 02:51:41.950950 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.950955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.951617 (+ 662us) server_negotiation.cc:407] Connection header received
0811 02:51:41.951824 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.951828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.951882 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.951938 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.952494 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57436: BlockingRecv error: recv got EOF from 127.0.0.1:57436 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 02:51:41.973203 29304 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:41.971262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57444 (local address 127.21.236.189:36581)
0811 02:51:41.971389 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:41.971394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:41.972079 (+ 685us) server_negotiation.cc:407] Connection header received
0811 02:51:41.972296 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:41.972300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:41.972359 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:41.972415 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:41.973073 (+ 658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57444: BlockingRecv error: recv got EOF from 127.0.0.1:57444 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 02:51:42.009724 29304 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:42.008293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57448 (local address 127.21.236.189:36581)
0811 02:51:42.008443 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:42.008448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:42.008725 (+ 277us) server_negotiation.cc:407] Connection header received
0811 02:51:42.008927 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:42.008931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:42.008991 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:42.009089 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:42.009592 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57448: BlockingRecv error: recv got EOF from 127.0.0.1:57448 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 02:51:42.077220 29304 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:42.075987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57452 (local address 127.21.236.189:36581)
0811 02:51:42.076175 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:42.076178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:42.076297 (+ 119us) server_negotiation.cc:407] Connection header received
0811 02:51:42.076469 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:42.076472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:42.076529 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:42.076626 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:42.077117 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57452: BlockingRecv error: recv got EOF from 127.0.0.1:57452 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 02:51:42.209206 29304 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:42.207807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57454 (local address 127.21.236.189:36581)
0811 02:51:42.207952 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:42.207959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:42.208056 (+ 97us) server_negotiation.cc:407] Connection header received
0811 02:51:42.208278 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:42.208283 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:42.208341 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:42.208453 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:42.208998 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57454: BlockingRecv error: recv got EOF from 127.0.0.1:57454 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 02:51:42.469542 29304 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 02:51:42.468167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57470 (local address 127.21.236.189:36581)
0811 02:51:42.468335 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:42.468341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:42.468534 (+ 193us) server_negotiation.cc:407] Connection header received
0811 02:51:42.468724 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:42.468728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:42.468792 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:42.468895 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 02:51:42.469425 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57470: BlockingRecv error: recv got EOF from 127.0.0.1:57470 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 02:51:42.773705 29505 catalog_manager.cc:1259] Loaded cluster ID: 79628256fff5443a9a62579f313eef56
I20260811 02:51:42.773818 29505 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc: loading cluster ID for follower catalog manager: success
I20260811 02:51:42.775642 29505 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc: acquiring CA information for follower catalog manager: success
I20260811 02:51:42.776963 29505 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6e923cfc211e4b6ca01a2a469cb9e8dc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:51:42.777822 29489 catalog_manager.cc:1259] Loaded cluster ID: 79628256fff5443a9a62579f313eef56
I20260811 02:51:42.777881 29489 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d: loading cluster ID for follower catalog manager: success
I20260811 02:51:42.779592 29489 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d: acquiring CA information for follower catalog manager: success
I20260811 02:51:42.780781 29489 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c6891dee07af42b481cb226c1c25548d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 02:51:42.988425 29665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:42.984101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57480 (local address 127.21.236.189:36581)
0811 02:51:42.984376 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:42.984380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:42.984504 (+ 124us) server_negotiation.cc:407] Connection header received
0811 02:51:42.984702 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:42.984704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:42.984838 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:42.984990 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:42.985286 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:42.985819 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:42.986753 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:42.986993 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:42.987139 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:42.987510 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:42.987976 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:42.988120 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:42.988187 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 02:51:42.988250 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":73,"threads_started":1}
I20260811 02:51:42.995966 29675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:42.992024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36708 (local address 127.21.236.188:41101)
0811 02:51:42.992310 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:42.992315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:42.992331 (+ 16us) server_negotiation.cc:407] Connection header received
0811 02:51:42.992461 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:42.992464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:42.992584 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:42.992728 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:42.993000 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:42.993520 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:42.994389 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:42.994628 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:42.994747 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:42.995106 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:42.995551 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:42.995705 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:42.995744 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 02:51:42.995802 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":77,"threads_started":1}
I20260811 02:51:43.004647 29665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:43.000100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57494 (local address 127.21.236.189:36581)
0811 02:51:43.000230 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:43.000234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:43.000740 (+ 506us) server_negotiation.cc:407] Connection header received
0811 02:51:43.000881 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:43.000885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:43.000945 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:43.001017 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:43.001403 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:43.002311 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:43.003365 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:43.003584 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:43.003738 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:43.004039 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:43.004152 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:43.004226 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:43.004374 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 02:51:43.004432 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 02:51:43.004807 29675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:43.000190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36718 (local address 127.21.236.188:41101)
0811 02:51:43.000305 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:43.000310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:43.000324 (+ 14us) server_negotiation.cc:407] Connection header received
0811 02:51:43.000500 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:43.000504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:43.000552 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:43.000636 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:43.000931 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:43.001843 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:43.003266 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:43.003573 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:43.003664 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:43.003968 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:43.004063 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:43.004132 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:43.004615 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 02:51:43.004681 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 02:51:43.005844 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 02:51:42.999770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54500 (local address 127.21.236.190:40665)
0811 02:51:43.000071 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 02:51:43.000077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 02:51:43.000094 (+ 17us) server_negotiation.cc:407] Connection header received
0811 02:51:43.000338 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 02:51:43.000344 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 02:51:43.000524 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 02:51:43.000649 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 02:51:43.001069 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:43.001853 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:43.003773 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 02:51:43.004158 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 02:51:43.004286 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 02:51:43.004789 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 02:51:43.005217 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 02:51:43.005337 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 02:51:43.005445 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 02:51:43.005510 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":107,"threads_started":1}
I20260811 02:51:43.010354 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 29049
I20260811 02:51:43.027091 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 29176
I20260811 02:51:43.039546 22450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQxaM5w/build/debug/bin/kudu with pid 29309
2026-08-11T02:51:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3497 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (7044 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (307359 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST